An Open Letter to NIST About SP 800-30

Posted June 9th, 2008 by

Dear NIST People,

I have this semi-random digital scribbling thingie called a blog.  You might have heard of them.  Hey, you might have even at one point heard of mine.  =)

On my blog I let it be known that I am what the rest of the world would call a “NIST Cheerleader”.  I watch your every move.  I comment on your new publications.  I teach your framework every quarter.  From time to time, I criticize, but only because I have a foot in the theory of information security that you live and a foot in the implementation with agencies who know where the theory and models break.

The best thing that you have given us is not the risk management framework, it was SP 800-30, “Risk Management Guide for Information Systems”.  It’s small, to-the-point, and scalable from a single server to an entire IT enterprise.  Sure, the quants hate it, but for the quals and Government, it’s good enough.  I know private-sector organizations that use it.  One of my friends and blog readers/commenters was the guy who taught a group of people how to do risk assessment, then these same people went on to help you write the book.

I heard that you were in the process of revising SP 800-30.  While this is much needed to catch up/modernize, I want to make sure that 800-30 does not follow the “live by the catalog, die by the catalog” path that we seem to be following lately.  In other words, please don’t change risk assessment process to the following:

  1. Determine boundary
  2. Determine criticality
  3. Conduct a gap assessment against a catalog of controls (SP 800-53/800-53A)
  4. Attach a priority to mitigation
  5. Perform risk avoidance because compliance models are yes/no frameworks
  6. Document
  7. ???
  8. Profit!

Use at your own risk.  Play safely, have fun!

At Your Own Risk Photo by  Mykl Roventine.

The reason that I am writing this is to let you know that I have noticed a disturbing trend in how now that we have a catalog of controls, the risk management framework is focusing more and more heavily on the catalog as the vehicle for determine an adequate level of security.  Some of this is good, some of this is not.

Why am I so concerned about this?  Well, inside the Government we have 2 conflicting ideas on information security:  compliance v/s risk management.  While we are fairly decent Government-wide at compliance management, the problem that we have is in risk management because risk management is only as good as the people who perform the risk assessment.  Not that we don’t have competent people, but the unknowns are what will make or break your security program, and the only way that you can known the unknowns is to get multiple assessments aimed at risks outside of the control catalog.

However, if you change the risk assessment process to a “catalog of controls gap analysis” process, then we’ve completely lost risk management in favor of compliance management.  To me, this is a disturbing trend that needs to be stopped.

Thank you for your time


Similar Posts:

Posted in FISMA, NIST, Rants, Risk Management | 10 Comments »

10 Responses

  1.  Vlad the Impaler Says:


    Well written. I just hope that Dr. Ron and the other good folks at NIST aren’t limited in hearing this opinion soleley from your blog…


  2.  Jeremy Says:

    Just started reading your blog today. You have a lot of good information and thoughts on here.

    Do you have an answer to the balance between Compliance and Risk Management?

    This is something I struggled with the day I started doing C&A. Just because you have a piece of paper, doesn’t mean you are secure.

    Here is the problem a lot of security analysts are put in. The client (government agency) just wants the passing grade. They don’t want to know all the little ins and outs of how and why. And in fact, they will argue with you that because they can put a check mark in the box, they passed.

    For example, an agency may have a policy regarding media protection (hopefully they all do at this point). But, it may be utter junk. If the analyst points that out, the client states I don’t care. We have the policy. It passes. How do you propose changing that way of thinking? It has to be from the top down, I would think. Is this just part of the growing pains of information security maturity?

  3.  rybolov Says:

    Hi Jeremy

    Sadly, this is a fairly common problem. The only thing that really changes the situation is education and eventually changing the culture of the Government from being compliance-centric to risk management. Neither is going to happen in a couple of weeks.

    There are some things that help significantly, probably the best at your level is to capture all vulnerabilities in a POA&M. That way, even if the system is fully accredited, you have tracking to get things fixed.

    The implied task here is that you get good vulnerability assessments and assessments from different groups over time.

  4.  Jeremy Says:

    So, what can a lowly security consultant do without getting fired to educate and change the federal government?

    As a small security company who struggles to find business against the SAIC’s, GD’s and L3’s of the world, it is a delicate balancing act to do the right thing and still stay employeed.

    It seems to me that the financial side of things is just more mature. An auditor can stand up and say, look this is not good enough (unless you are Arthur Anderson). No one is going to fire their financial auditor, that I know of.

    In the C&A world, this does not seem to be true. Maybe it is at the large security consulting firms?

  5.  Darren Couch Says:

    In my tiny slice of the federal system I work in the same thinking is also true. Or, even worse, the people they hire and train to do risk analysis and implement it take off the minute they have their “credentials” in hand to the private sector. So, the system just keeps rolling along the way it always did until a flurry of activity happens to whatever threat du jour makes its appearance. It is certainly a frustration, but I agree that time and patience (and in the case of the military side, a little incentive to keep the talent at home) will eventually win.

  6.  Dan Philpott Says:


    Finding the balance between compliance and risk management? Risk management is the game, compliance provides the statistics. As in real life you can play fantasy security using statistics and mental gymnastics, but don’t mistake that for real security.

    So what’s the point of compliance? In the particular it tells federal managers how well they meet a set of standards and what they need to improve. In the aggregate it tells them where they are in relation to others and where they should improve. In the right hands it is a mirror of reality and leverage to institute improvements. In the wrong hands it’s a checklist.

    What are the right hands? The right hands are the ones attached to someone who can effectively communicate to managers. Communicating compliance information is a complicated process. You can drop it like a fait accompli rock on the head of a manager or you can weave your findings and concerns into the fabric of an organization. It all depends on the context of the consultant’s environment which extreme is preferred. What it should always be is a dialog between you as the consultant, with the expertise and perspective, and them as a client, with the systems and security needs.

    What does communication have to do with anything? Well, everything. See, agencies don’t want anything. People in agencies want things. And people don’t want one thing, they want a basket of related things. If they want a passing grade in compliance they have other wants. They may want better security. They may want to avoid a front page, above the fold, Washington Post article about their organization. They may want the admiration of their peers. They may want to make the IG happy. They may want a bigger budget. They may want a larger head count. Good communications matches the security needs to that person’s wants.

    Saying how to most effectively communicate information is far outside the bounds of this reply but I can’t say enough good things about Stanley Bing (Throwing The Elephant comes to mind) and Edward Tufte.

    But this isn’t SECURITY! If you want security to happen then this is security. Just as army brass massage congress critters for budget dollars, so must security consultants genuflect to IT managers for improved security.

    Have you noticed I love answering my own questions and exclamations!?

  7.  The Way Not to Change NIST SP 800-30 | Says:

    […] infosec NoVA-based blog, has put together a great blog post about NIST’s latest effort to modernize SP 800-30: Risk Management Guide for Information Systems. In his post he stresses how NIST should not change this document into a “catalog of controls […]

  8.  System Advancements at the Monastery » Blog Archive » Risk Assessment: A Starting Point Says:

    […] Smith, the Guerilla CISO, had a posting “An Open Letter to NIST About SP 800-30“. Michael writes “The best thing that you have given us is not the risk management […]

  9.  Gary Stoneburner Says:

    Hi all,

    If anything, SP 800-30 Rev 1 will stress risk management more, not less.

    SP 800-30 Rev 1 will be expressed as part of the risk management process to be described in SP 800-39. A process where controls are a means, not the purpose.

    The focus is managing the information system-related risk arising from our dependence on information systems in an operational environment of competent cyber attackers. We manage these risks in order to be able to achieve mission/business success and to do so without unacceptable damage to organizations, individuals, and the Nation.

    We don’t just count controls; we measure risk to be able to manage it.

  10.  Mike Nelson Says:

    Gary… we’re getting close to the target Initial Public Draft phase for SP800-30 Rev 1. Any more insights you can provide into how it will evolve? I agree with rybolov’s now over a year old observations (and as always, awed by Mr. Philpott’s eloquence). Please try to give us a risk assessment approach that serves as a TOOL for the effective communication of real risk in the proper context. Are you seeing any value in the State Department’s Risk Scoring approach?

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

Visitor Geolocationing Widget: