[SCADASEC] The Difference Between Audits, Assessments, and Analysis

[Clint Bodungen]

This is a message that I posted on the other SCADA thread in response to Joe
(no offense, Joe) that I think is important enough to cross over to this
list.  It just happened to prompt something that has bugged me for a while
in this industry and I think it is very important that we all have the same
understanding of this subject.

It seems that there is one term being used here to describe two completely
different procedures.  The term "Audit" is used to describe a process that
is measurable and legally enforceable by regulated standards in which there
is a penalty for non-compliance.  If I am understanding you correctly here,
Joe, you are referring to an assessment or possibly even an analysis. 

If you are looking for a metrics or set of standards that will help
measure/assess the current state of your industrial control systems, this
would describe an assessment; and then, possibly, analyze the risk exposure
based on conditional formulas, which would describe an analysis.  

If you are talking about a measurement against a set of regulated standards
that can be legally enforced with penalties for non-compliance, then this is
an audit.  Technically, NERC CIP can be audited against, and we are warned
it will be, because it is legally enforceable even though there is no real
metrics to measure against, just processes, as you eluded to.  This is
actually one of my fundamental problems with NERC CIP being a regulatory
*enforceable* standard.  

These three terms (Audit, Assessment, and Analysis) seem to be thrown around
interchangeably throughout this industry and they should not be as it causes
confusion both procedurally and legally.

For what it's worth... cheers.

Clint Bodungen   

P.S. - There is, of course, the other common use of "Audit", which refers to
event logging.  This is not the definition of the term I am talking about
here.



-----Original Message-----
From: scada-bounces at scadaperspective.com
[mailto:scada-bounces at scadaperspective.com] On Behalf Of Joe Weiss
Sent: Friday, February 15, 2008 10:09 AM
To: scada at scadaperspective.com
Subject: [SCADA] Control System Audits

One important point on auditing - I do not believe we have audit metrics
for securing industrial control systems. The NERC CIP process is just
that- a process. It is focused on paperwork not assuring the facility is
more secure. We need to develop audit metrics for industrial control
systems.
Joe

Joe Weiss PE, CISM
Applied Control Solutions, LLC
Cupertino, CA
(408) 253-7934
(408) 253-7974 Fax
(408) 832-5396 Cell
joe.weiss at realtimeacs.com