Indicator Species

Posted February 19th, 2007 by

Trout are an indicator species.  You can tell how healthy the stream is by counting the number of trout and the size of trout in a particular section.  Trout need clean water, a certain temperature range of water, protection from predators, unsilted gravel to spawn in, and a food supply like smaller fish and invertebrates.  So absence of trout means absence of these factors, which by extension means an unhealthy stream.

There are even metrics for this: number of trout per mile, pounds of trout per mile, average size of trout.  Biologists do periodic electroshocking surveys to capture the fish, weigh and measure them, then release them back into the current.  All in the interest of gathering metrics.

By extension, a very valuable tool for an information security manager is to be able to gather metrics.  Instead of trout per mile, we are interested in total number of vulnerabilities in our information system.  Instead of pounds of trout per mile, we are interested in the aggregate risk to our enterprise.  And so on.

Enter Certification and Accreditation.  It is not just a paperwork exercise.  There, I said it.  It is, however, risk assessment and the gathering of metrics to determine how well our security program is progressing (or not, as the case may be).

As a whole, the government is spending $FooMillions on certification and accreditation and still losing the battle.  I know one agency that is in the process of getting fleeced year after year by unscrupulous contractors selling C&A solutions.  It seems like everybody I’ve worked with previously on a project who didn’t have the skills to succeed is now being billed to this agency as a subject-matter expert.  For every 30 people the agency hires, they get 5 that are any good, and the 25 bad ones can mess things up faster than the others can fix them.

Why is C&A in such a pathetic state?

Well, this is apparently a little-known secret: C&A is an indicator, not the actual act of providing “adequate security”.  If a security program is in place and effective, then it’s relatively easy to satisfy C&A requirements but not the other way around–it is possible to have a certified and accredited system that does not provide adequate security.

With C&A getting such a high amount of press from the guardians of all things security (NIST, OMB, and GAO), what has happened down among the practitioners is that the focus has switched to the indicators instead of the root cause.  Going back to our trout stream, we’re expecting the pounds of trout per mile to go up based solely on the fact that we keep conducting electroshock surveys.

So how do we succeed at the information security game?  One of the steps is to realize C&A for what it is (a risk assessment and metrics tool for decision-makers, a method to incorporate security into the SDLC) and what it isn’t (a solution to internal agency politics, a comprehensive security program).  The next step is to relearn how to perform risk management, which is where the real intent and purpose of C&A lies.

Similar Posts:

Posted in FISMA, NIST, Risk Management, What Doesn't Work, What Works | 4 Comments »

4 Responses

  1.  The Guerilla CISO » Blog Archive » Reading Between the Letters G, A, and O Says:

    […] have to rely on metrics to give you a picture of how things are going, but at the end of the day, they’re still just that, indicators.  Of course, I haven’t worked with all 24 agencies, so maybe my worldview is pretty […]

  2.  The Guerilla CISO » Blog Archive » Metrics, Irrationality, Sports, and Malcolm Gladwell Says:

    […] security and because it’s dependent on so many external factors, it’s hard to point to one indicator to say “this one thing makes or breaks an information security […]

  3.  FISMA Report Card News, Formulas, and 3 Myths | The Guerilla CISO Says:

    […] back last year I wrote a blog post about indicator species and how we’re expecting the metrics to go up based on our continual measurin….  Every couple of months I go back and review it to see if it’s still relevant.  And the […]

  4.  Security Advancements at the Monastery » Blog Archive » FISMA Reform: Lieberman, Collins, and Carper Introduce Bill Says:

    […] the more likely the people being tested will be able to execute. This is highly wrong and I’ve commented on it before. I think that if it was really a fact of people being lazy or fraudulent then we would have fixed […]

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

Visitor Geolocationing Widget: