Posts RSS
Subscribe via Email
An Open Letter to NIST About SP 800-30
Posted June 9th, 2008 by rybolov
Dear NIST People,
I have this semi-random digital scribbling thingie called a blog. You might have heard of them. Hey, you might have even at one point heard of mine. =)
On my blog I let it be known that I am what the rest of the world would call a “NIST Cheerleader”. I watch your every move. I comment on your new publications. I teach your framework every quarter. From time to time, I criticize, but only because I have a foot in the theory of information security that you live and a foot in the implementation with agencies who know where the theory and models break.
The best thing that you have given us is not the risk management framework, it was SP 800-30, “Risk Management Guide for Information Systems”. It’s small, to-the-point, and scalable from a single server to an entire IT enterprise. Sure, the quants hate it, but for the quals and Government, it’s good enough. I know private-sector organizations that use it. One of my friends and blog readers/commenters was the guy who taught a group of people how to do risk assessment, then these same people went on to help you write the book.
I heard that you were in the process of revising SP 800-30. While this is much needed to catch up/modernize, I want to make sure that 800-30 does not follow the “live by the catalog, die by the catalog” path that we seem to be following lately. In other words, please don’t change risk assessment process to the following:
- Determine boundary
- Determine criticality
- Conduct a gap assessment against a catalog of controls (SP 800-53/800-53A)
- Attach a priority to mitigation
- Perform risk avoidance because compliance models are yes/no frameworks
- Document
- ???
- Profit!
At Your Own Risk Photo by Mykl Roventine.
The reason that I am writing this is to let you know that I have noticed a disturbing trend in how now that we have a catalog of controls, the risk management framework is focusing more and more heavily on the catalog as the vehicle for determine an adequate level of security. Some of this is good, some of this is not.
Why am I so concerned about this? Well, inside the Government we have 2 conflicting ideas on information security: compliance v/s risk management. While we are fairly decent Government-wide at compliance management, the problem that we have is in risk management because risk management is only as good as the people who perform the risk assessment. Not that we don’t have competent people, but the unknowns are what will make or break your security program, and the only way that you can known the unknowns is to get multiple assessments aimed at risks outside of the control catalog.
However, if you change the risk assessment process to a “catalog of controls gap analysis” process, then we’ve completely lost risk management in favor of compliance management. To me, this is a disturbing trend that needs to be stopped.
Thank you for your time
–Rybolov
Similar Posts:
Posted in FISMA, NIST, Rants, Risk Management | 
10 Comments »
Tags: 800-30 • auditor • blog • catalogofcontrols • comments • compliance • fisma • government • infosec • management • risk • security
June 10th, 2008 at 4:28 pm
Comrade,
Well written. I just hope that Dr. Ron and the other good folks at NIST aren’t limited in hearing this opinion soleley from your blog…
Cheers,
Vlad
June 11th, 2008 at 1:09 pm
Just started reading your blog today. You have a lot of good information and thoughts on here.
Do you have an answer to the balance between Compliance and Risk Management?
This is something I struggled with the day I started doing C&A. Just because you have a piece of paper, doesn’t mean you are secure.
Here is the problem a lot of security analysts are put in. The client (government agency) just wants the passing grade. They don’t want to know all the little ins and outs of how and why. And in fact, they will argue with you that because they can put a check mark in the box, they passed.
For example, an agency may have a policy regarding media protection (hopefully they all do at this point). But, it may be utter junk. If the analyst points that out, the client states I don’t care. We have the policy. It passes. How do you propose changing that way of thinking? It has to be from the top down, I would think. Is this just part of the growing pains of information security maturity?
June 12th, 2008 at 10:07 am
Hi Jeremy
Sadly, this is a fairly common problem. The only thing that really changes the situation is education and eventually changing the culture of the Government from being compliance-centric to risk management. Neither is going to happen in a couple of weeks.
There are some things that help significantly, probably the best at your level is to capture all vulnerabilities in a POA&M. That way, even if the system is fully accredited, you have tracking to get things fixed.
The implied task here is that you get good vulnerability assessments and assessments from different groups over time.
June 12th, 2008 at 10:35 am
So, what can a lowly security consultant do without getting fired to educate and change the federal government?
As a small security company who struggles to find business against the SAIC’s, GD’s and L3’s of the world, it is a delicate balancing act to do the right thing and still stay employeed.
It seems to me that the financial side of things is just more mature. An auditor can stand up and say, look this is not good enough (unless you are Arthur Anderson). No one is going to fire their financial auditor, that I know of.
In the C&A world, this does not seem to be true. Maybe it is at the large security consulting firms?
June 12th, 2008 at 11:03 am
In my tiny slice of the federal system I work in the same thinking is also true. Or, even worse, the people they hire and train to do risk analysis and implement it take off the minute they have their “credentials” in hand to the private sector. So, the system just keeps rolling along the way it always did until a flurry of activity happens to whatever threat du jour makes its appearance. It is certainly a frustration, but I agree that time and patience (and in the case of the military side, a little incentive to keep the talent at home) will eventually win.
June 12th, 2008 at 4:04 pm
Jeremy,
Finding the balance between compliance and risk management? Risk management is the game, compliance provides the statistics. As in real life you can play fantasy security using statistics and mental gymnastics, but don’t mistake that for real security.
So what’s the point of compliance? In the particular it tells federal managers how well they meet a set of standards and what they need to improve. In the aggregate it tells them where they are in relation to others and where they should improve. In the right hands it is a mirror of reality and leverage to institute improvements. In the wrong hands it’s a checklist.
What are the right hands? The right hands are the ones attached to someone who can effectively communicate to managers. Communicating compliance information is a complicated process. You can drop it like a fait accompli rock on the head of a manager or you can weave your findings and concerns into the fabric of an organization. It all depends on the context of the consultant’s environment which extreme is preferred. What it should always be is a dialog between you as the consultant, with the expertise and perspective, and them as a client, with the systems and security needs.
What does communication have to do with anything? Well, everything. See, agencies don’t want anything. People in agencies want things. And people don’t want one thing, they want a basket of related things. If they want a passing grade in compliance they have other wants. They may want better security. They may want to avoid a front page, above the fold, Washington Post article about their organization. They may want the admiration of their peers. They may want to make the IG happy. They may want a bigger budget. They may want a larger head count. Good communications matches the security needs to that person’s wants.
Saying how to most effectively communicate information is far outside the bounds of this reply but I can’t say enough good things about Stanley Bing (Throwing The Elephant comes to mind) and Edward Tufte.
But this isn’t SECURITY! If you want security to happen then this is security. Just as army brass massage congress critters for budget dollars, so must security consultants genuflect to IT managers for improved security.
Have you noticed I love answering my own questions and exclamations!?
June 16th, 2008 at 10:30 am
[…] infosec NoVA-based blog, has put together a great blog post about NIST’s latest effort to modernize SP 800-30: Risk Management Guide for Information Systems. In his post he stresses how NIST should not change this document into a “catalog of controls […]
September 22nd, 2008 at 2:11 am
[…] Smith, the Guerilla CISO, had a posting “An Open Letter to NIST About SP 800-30“. Michael writes “The best thing that you have given us is not the risk management […]
March 14th, 2009 at 12:32 am
Hi all,
If anything, SP 800-30 Rev 1 will stress risk management more, not less.
SP 800-30 Rev 1 will be expressed as part of the risk management process to be described in SP 800-39. A process where controls are a means, not the purpose.
The focus is managing the information system-related risk arising from our dependence on information systems in an operational environment of competent cyber attackers. We manage these risks in order to be able to achieve mission/business success and to do so without unacceptable damage to organizations, individuals, and the Nation.
We don’t just count controls; we measure risk to be able to manage it.
July 13th, 2009 at 10:31 pm
Gary… we’re getting close to the target Initial Public Draft phase for SP800-30 Rev 1. Any more insights you can provide into how it will evolve? I agree with rybolov’s now over a year old observations (and as always, awed by Mr. Philpott’s eloquence). Please try to give us a risk assessment approach that serves as a TOOL for the effective communication of real risk in the proper context. Are you seeing any value in the State Department’s Risk Scoring approach?