[SCADASEC] Regarding "Bandolier"

Sat May 31 15:36:47 CDT 2008

So I'll chime in here...

CAVEAT: I do work for Tenable, the folks that make Nessus, but I'll
try to avoid any marketing speak

1) Your vulnerability scanner (or IDS sensor) typically provides
information on threats or vulnerabilities that have predetermined
severity levels and in some cases a CVSS score that is NOT
site/installation specific.

For example, for a default Modicon HTTP password we see that it is critical

http://www.nessus.org/plugins/index.php?view=single&id=23822

The problem of assigning meaningful risk scores to vulnerabilities in
different environments is non-trivial but if you want to "go there"
see http://www.first.org/cvss/cvss-guide.html

2) The point of the blog on "compliance scan" results (like those
generated by Bandolier but this would apply to the FDCC or CIS
compliance checks in Nessus 3) is that that they do NOT have
severities. Because the Nessus 3 compliance checks check for known
good configurations (typically based on standards, benchmarks, in
organizations best practices) they are conditions which are either
true or not. You are either compliant or not -- or you don't know.

One might disagree with the standard about services running on a
workstation or registry settings or whatever, but these results are
not relative to your site, risk level, etc.

3) But back to the vulnerability severity (or "risk factor" as it is
labeled in Nessus results) that Joe brings up, this is where your
vulnerability management policy, strategy, and your vulnerability
workflow toolset comes into play. I don't know how folks do this with
Remedy or other ticketing systems, but Tenable Security Center allows
you to reclassify the risk based on site/installation/organizational
specific considerations. You can downgrade/upgrade the severity of
vulnerabilities. When vendors release vulnerabilities, when IDS/IPS
(or scanner) vendors adopted CVSS they attempted to come up with a
best guess at this, but it is impossible to come up with a score that
makes everybody happy. When folks get vulns from their vendor (or a
CERT or a service like Secunia) you obviously have to evaluate the
specific vulnerability in your environment.

4) I don't want to stir up the "value of compliance and does it mean
you are secure" (sure I do) the debate, but to me the issue of impact
of a given vulnerability is different from whether a given system is
compliant or not. These seem to be conflated. Also, vulnerability
scanning and compliance monitoring tools don't directly measure or
address organizational policies only the *technical implementation* of
those policies. The look at the state of systems and networks. You
might be able to make some inferences about security policies or
architecture based on scan results or analysis of logs, but that is a
different problem space.

5) I would agree with Jason that false positives are more a concern
with IDS and Network Vulnerability scanning results than in compliance
checks. The Bandolier compliance (like all of the Nessus compliance)
checks actually log into the remote system using SSH/SMB/WMI to
collect data about the systems to compare to the audit policy. In
general, local checks are far less prone to false positives because
they are querying patch levels, services, files content and
permissions as a priveleged user vs. an unauthenticated scan which
would definitely be more likely to have false positives depending the
responses the scanners is able to elicit from the target.

- mdf

> Hi,
>
> Usually just a lurker here but figured it was time for my first post
> considering my relationship to this topic. I'm the technical lead for
> Bandolier and will do my best to answer some questions about the project.
>
> ------------------------
> 1.) "My question is this, and a rather simple one: is it possible to
> have a vulnerabilities checker (such as Bandolier) to consistently find
> "anti-problems"?  Meaning...that Bandolier would find only the areas
> that we already know about and those that we don't?"
> ------------------------
> Good question and the short answer is no -- but Bandolier is really not
> about finding the "anti-problems" or even checking vulnerabilities. The
> goal of Bandolier is to add control system application intelligence to a
> feature in Nessus (and other scanners in the future) known as compliance
> checks or audit files. The simplest way to describe it is with the
> "known bad/known good" concept. The Bandolier project has more to do
> with establishing a "known good" configuration for the selected control
> system applications and validating that it hasn't changed over time --
> as opposed to vulnerability scanning which helps identify the "known bad".
>
> ------------------------
> 2.) "...what does the Bandolier product do with regards to "false
> positives"?"
> ------------------------
> The result of an audit check is either compliant, non-compliant, or
> inconclusive. Whether false positives exist may depend on how you define
> the term. Here's an example:
>
> Let's say our best practice audit file for a control system app on Linux
> has a password policy that requires a length of at least eight
> characters with some complexity requirements (e.g. upper/lower/special
> characters). If your local security policy favors length instead of
> complexity (e.g. 20 character length, no upper/lower/special character
> requirement), then the audit check that looks for password complexity
> will report a non-compliant result. This could be considered a "false
> positive" but is probably different that how we usually think of false
> positives in vulnerability scanning and IDS. (Side note: The audit files
> are simple, plain text files that can be customized to an assess owner's
> security policy or requirements)
>
> For more information about Bandolier, please check out these links:
>
>    http://www.scadapedia.com/index.php/Bandolier
>
> http://www.digitalbond.com/index.php/2008/02/19/bandolier-update-introduction-to-compliance-checks/
>
> Hope that helps...
>
> Best Regards,
> Jason
>
> --
> Jason Holcomb
> Security Consultant and Researcher
> Digital Bond, Inc.
> holcomb at digitalbond.com <mailto:holcomb at digitalbond.com>
>
>
>
> Bob Radvanovsky wrote:
>> Something that I found unusual was a summary provided by Dale Peterson at the recent ieRoadmap conference here in Chicago, IL just 2 days ago.  He gave a presentation about a new product that was funded by DOE money.  The product is called "Bandolier".  Ironically, I found this on the "Unfettered" blog on the Control Global magazine's posting this morning.  Ironic?  Perhaps.
>>
>> But...what I found most interesting was something that Joe Weiss had pointed out:
>>
>> "... It seems to be a good approach to identifying vulnerabilities in control system computers. The severity ratings for Bandolier are a good idea but the approach does not go far enough. Since these ratings are used for compliance reporting, it potentially could cost companies a significant amount of money without an accompanying risk reduction. The missing piece is the impact on the process or facility. I see two issues with the Bandolier approach- the first is the classification of non-critical computers as "severe". The second is how Bandolier is used in the overall context of securing the facility. Many of the Major and Moderate control system cyber incidents I have identified in my incident database would not have been identified using an approach like Bandolier as they were not caused by traditional computer vulnerabilities but represented failures, omissions, or errors in design, configuration, or implementation of required programs and policies."
>>
>> My question is this, and a rather simple one: is it possible to have a vulnerabilities checker (such as Bandolier) to consistently find "anti-problems"?  Meaning...that Bandolier would find only the areas that we already know about and those that we don't?  Second, what does the Bandolier product do with regards to "false positives"?  A "false positive" is where an automated/semi-automated vulnerabilities checkers finds a problem, when there really is no problem at all (nor ever was a problem, either).  Without knowing more about this product, or seeing firsthand what this product does, I'd pose these 2 questions about the functionality of the Bandolier product.
>>
>> And last, but not least, doesn't Industrial Defender (a.k.a. Verano) have (or had) a product that does something similar?
>>
>> I pose this question to the list openly...come one and all.  I could be wrong, but I think oe might be onto something here.
>>
>> -rad
>>
>> DISCLAIMER:  This is merely an observational commentary from what I received -- in summary -- at the ieRoadmap conference these past 2 days.  In Dale's defense (as well as those who presented), they had 30 minutes to give a presentation, of which 15 minutes reserved for the actual presentation, the other 15 minutes were reserved for Q&A from the Roadmap WG and conference attendees.
>>
>> _______________________________________________
>> To unsubscribe from this mailing list, please visit:
>> http://news.infracritical.com/mailman/listinfo/scadasec
>>
>> To review our privacy statement, please visit:
>> http://www.infracritical.com/privacy.html
>>
>> scadasec at news.infracritical.com
>> http://news.infracritical.com/mailman/listinfo/scadasec
>
> <mailto:holcomb at digitalbond.com> <mailto:holcomb at digitalbond.com>
>
> _______________________________________________
> To unsubscribe from this mailing list, please visit:
> http://news.infracritical.com/mailman/listinfo/scadasec
>
> To review our privacy statement, please visit:
> http://www.infracritical.com/privacy.html
>
> scadasec at news.infracritical.com
> http://news.infracritical.com/mailman/listinfo/scadasec

-- 
Matthew Franz
http://www.threatmind.net/