Indicator Species

February 19th, 2007 by rybolov

If you're new here and would like to see more of what I'm saying, you may want to subscribe to my RSS feed (I can even email my blog posts to you when I publish a new one) or have a look at my papers and presentations page for downloads of stuff that you can share or "borrow heavily from". You also might find my guidelines for posting comments interesting, especially if you're a government employee. If you want to see me blog about anything in particular, drop me a private email on how you think I'm completely full of myself, extend me an invitation to speak at your next security meeting/event, or just to ship a huge bag of money in my direction, you can do that through my contact page. Thanks for visiting and happy hacking!

Trout are an indicator species.  You can tell how healthy the stream is by counting the number of trout and the size of trout in a particular section.  Trout need clean water, a certain temperature range of water, protection from predators, unsilted gravel to spawn in, and a food supply like smaller fish and invertebrates.  So absence of trout means absence of these factors, which by extension means an unhealthy stream.

There are even metrics for this: number of trout per mile, pounds of trout per mile, average size of trout.  Biologists do periodic electroshocking surveys to capture the fish, weigh and measure them, then release them back into the current.  All in the interest of gathering metrics.

By extension, a very valuable tool for an information security manager is to be able to gather metrics.  Instead of trout per mile, we are interested in total number of vulnerabilities in our information system.  Instead of pounds of trout per mile, we are interested in the aggregate risk to our enterprise.  And so on.

Enter Certification and Accreditation.  It is not just a paperwork exercise.  There, I said it.  It is, however, risk assessment and the gathering of metrics to determine how well our security program is progressing (or not, as the case may be).

As a whole, the government is spending $FooMillions on certification and accreditation and still losing the battle.  I know one agency that is in the process of getting fleeced year after year by unscrupulous contractors selling C&A solutions.  It seems like everybody I’ve worked with previously on a project who didn’t have the skills to succeed is now being billed to this agency as a subject-matter expert.  For every 30 people the agency hires, they get 5 that are any good, and the 25 bad ones can mess things up faster than the others can fix them.

Why is C&A in such a pathetic state?

Well, this is apparently a little-known secret: C&A is an indicator, not the actual act of providing “adequate security”.  If a security program is in place and effective, then it’s relatively easy to satisfy C&A requirements but not the other way around–it is possible to have a certified and accredited system that does not provide adequate security.

With C&A getting such a high amount of press from the guardians of all things security (NIST, OMB, and GAO), what has happened down among the practitioners is that the focus has switched to the indicators instead of the root cause.  Going back to our trout stream, we’re expecting the pounds of trout per mile to go up based solely on the fact that we keep conducting electroshock surveys.

So how do we succeed at the information security game?  One of the steps is to realize C&A for what it is (a risk assessment and metrics tool for decision-makers, a method to incorporate security into the SDLC) and what it isn’t (a solution to internal agency politics, a comprehensive security program).  The next step is to relearn how to perform risk management, which is where the real intent and purpose of C&A lies.

Posted in FISMA, NIST, Risk Management, What Doesn't Work, What Works |

3 Responses

  1. The Guerilla CISO » Blog Archive » Reading Between the Letters G, A, and O Says:

    [...] have to rely on metrics to give you a picture of how things are going, but at the end of the day, they’re still just that, indicators.  Of course, I haven’t worked with all 24 agencies, so maybe my worldview is pretty [...]

  2. The Guerilla CISO » Blog Archive » Metrics, Irrationality, Sports, and Malcolm Gladwell Says:

    [...] security and because it’s dependent on so many external factors, it’s hard to point to one indicator to say “this one thing makes or breaks an information security [...]

  3. FISMA Report Card News, Formulas, and 3 Myths | The Guerilla CISO Says:

    [...] back last year I wrote a blog post about indicator species and how we’re expecting the metrics to go up based on our continual measurin….  Every couple of months I go back and review it to see if it’s still relevant.  And the [...]

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.


Visitor Geolocationing Widget: