OK, now for the control-by-control analysis of the 20 Critical Security Controls.  This is part 2.  Look here for the first installment.  Read part 3 here.

Critical Control 1: Inventory of Authorized and Unauthorized Devices. This is good: get an automated tool to do IT asset discovery.  Actually, you can combine this with Controls 2, 3, 4, 11, and 14 with some of the data center automation software–you know the usual suspects, just ask your ops folks how you get in on their tools.  This control suffers from scope problems because it doesn’t translate down to the smaller-system scale:  if I have a dozen servers in an application server farm inside of a datacenter, I’ll usually know if anybody adds something.  The metric here (detect all new devices in 24 hours) “blows goats” because you don’t know if you’re detecting everything.  A better test is for the auditor to do their own discovery scans and compare it to the list in the permanent discovery tool–that would be validation that the existing toolset does work–with a viable metric of “percentage of devices detected on the network”.  The 24 hour metric is more like a functional requirement for an asset discovery tool.  And as far as the isolation of unmanaged assets, I think it’s a great idea and the way things should be, except for the fact that you just gave us an audit requirement to implement NAC.

Critical Control 2: Inventory of Authorized and Unauthorized Software. Sounds like the precursor to whitelisting.  I think this is more apropos to the Enterprise unless your system is the end-user computing environment (laptops, desktops).  Yes, this control will help with stuff in a datacenter to detect when something’s been pwned but the real value is out at the endpoints.  So yes, not happy with the scope of this control.  The metric here is as bad as for Control 1 and I’m still not happy with it.  Besides, if you allow unauthorized software to be on an IT device for up to 24 hours, odds are you just got pwned.  The goal here should be to respond to detected unauthorized software within 24 hours.

Critical Control 3: Secure Configurations for Hardware and Software on Laptops, Workstations, and Servers. This is actually a good idea, provided that you give me a tool to apply the settings automagically because manual configuration sucks.  I think it’s about a dozen different controls all wrapped into one, it’s just trying to do too much in one little control.  The time-based metric for this control is really bad, it’s like watching a train wreck.  But hey, I’ll offer up my own: percentage of IT assets conforming to the designated configuration.  It’s hinted at in the implementation guide, make it officially the metric and this might be a control I can support.

Critical Control 4: Secure Configurations for Network Devices such as Firewalls, Routers, and Switches. This is basically Control 3 for network devices.  The comments there also apply here.

Critical Control 5: Boundary Defense. This control is too much stuff crammed into one space.  As a result, it’s not concise enough to be implemented–it’s all over the map.  In fact, I’ll go as far as to say that this isn’t really one control, it’s a control theme with a ton of controls inside of it.  The “audit requirements” here are going to utterly kill me as a security manager because there is so much of a disparity between the control and the actual controls therein.

Critical Control 6: Maintenance, Monitoring, and Analysis of Audit Logs. Some of this control should be part of Controls 3 and 4 because, let’s be honest here, you’re setting up logging on devices the way that the hardening guide says you should.  The part that’s needed in this control is aggregation of logs and review of logs–get them off all the endpoints and into a centralized log management solution.  This is mentioned as the last “advanced” implementation technique but if you’re operating a modern Enterprise, I don’t see how you can get the rest of the implementation done without some kind of SIEM piece.   I just don’t get the metric here, again with the 24 hours.  How about “percentage of devices reporting into the SIEM”?  Yeah, that’s the easy money here.  The testing of this control makes me do a facepalm:  “At a minimum the following devices must be tested: two routers, two firewalls, two switches, ten servers, and ten client systems.”  OK, we’ve got a LAN/WAN with 15000 endpoints and that’s all we’re going to test?

Critical Control 7: Application Software Security. You keep using those words, I do not think they mean what you think they mean.  Application security is a whole different world and 20 CSC doesn’t even begin to scratch the surface of it.  Oh, but guess what?  It’s a tie-in to the 25 Most Dangerous Programming Errors which is about all this control is:  a pointer to a different project.  The metric here is very weak because it’s not tied back to the actual control.

Critical Control 8: Controlled Use of Administrative Privileges. This should be part of Controls 3 and 4, along with something about getting an Identity and Access Management system so that you have one ID repository.  I know this is a shocker to you, but the metric here sucks.

Critical Control 9: Controlled Access Based on Need to Know. This is a great idea, but as a control it’s too broad to achieve, which is why the 20 CSC were created in the first place.  What do we really want here?  Network share ACLs are mentioned, which is a control in itself, but the rest of this is hazy and leaves much room for interpretation.  Cue “audit requirements” and the part where Rybolov says “If it’s this hazy, it’s not really a standard, it’s a guideline that I shouldn’t be audited against.

Critical Control 10: Continuous Vulnerability Assessment and Remediation. All-in-all, not too bad.  I would suggest “Average time to resolve scan findings” here as a metric or even something as “hoakey” as the FoundScan metric just to gauge overall trends.

Arm Control photo by Crotchsplay.

Critical Control 11: Account Monitoring and Control. Haven’t we seen this before?  Yep, this should be incorporated into Controls 8, 3, and 4.  However, periodic account reviews are awesome if you have the patience to do it.

Critical Control 12: Malware Defenses. OK, this isn’t too bad.  Once again, the metric sucks, but I do like some of the testing steps.  The way I would test this is to compare our system inventory with my total list of devices.  A simple diff later, we have a list of unmanaged devices.

Critical Control 13: Limitation and Control of Network Ports, Protocols, and Services. Host firewalls was not what I thought of… I’m thinking more like firewalls and network segmentation where you have to get change control approval to add a firewall rule.  As far as the host setup, this should be part of Control 3.

Critical Control 14: Wireless Device Control. Not bad, but this should be dumped into a technical standard that you use like a hardening guide.  Metric here still sucks, but I don’t really need to say this again… oh wait, I just did.

Critical Control 15: Data Loss Prevention. Puh-lease.  I’ll be the first to admit, I’m a big believer in DLP done right, and that it’s an awesome tool to solve some of the unique .  But I don’t think that the market is mature enough to add it into your catalog of controls.  Also this will fall flat on its face if your system is just a web application cluster:  DLP addresses the endpoints (desktops, laptops, mobiles) and the outbound gateways (email, web, etc).  The problem with this control is that if you don’t buy and implement a full DLP solution (cue Rich Mogull and his DLP guide), there isn’t anything else that has a similar capability.  This is one of those controls where the 800-53 mapping gets really creative–Good Ship Lollipop Creative because we’re tapdancing around the issue that DLP-type solutions aren’t specifically required in 800-53.

These controls don’t have automated ways to implement and test them:

Critical Control 16: Secure Network Engineering. This control is a steaming crater.  It’s very much a guideline instead of an auditable standard.

Critical Control 17: Penetration Tests and Red Team Exercises. Not bad.  Still too easy to shop around for the bargain-basement penetration test team.  But yeah, pretty good overall.

Critical Control 18: Incident Response Capability. Good control.  Hard to test/audit except to look at after-incident reports.

Critical Control 19: Data Recovery Capability. Not bad here.  Not real COOP/DR/ITCP but about on par with typical controls frameworks.

Critical Control 20: Security Skills Assessment and Appropriate Training to Fill Gaps. Good idea.  Hard to implement without something like 8570.10 to give you a matrix by job position.  You want to change the world here, give your own mapping in the control.

Bookmark to:

Hide Sites

This is a multi-post series.  You are at post 1.  Read post 2 here.  Read Post 3 here.

This post begins with me.  For the past hour or so I’ve been working on a control-by-control objective analysis of SANS 20 Critical Security Controls.  This is a blog post I’ve had sitting “in the hopper” for 9 months or so.  And to be honest, I see some good things in the 20 CSC literature.  I think that, from a holistic perspective, the 20 CSC is an attempt at creating a prioritization of this huge list of stuff that I have to do as an information security officer–something that’s really needed.  I go into 20 CSC with a very open mind.

Then I start reading the individual controls.  I’m a big believer in Bottom-Line-Up-Front, so let me get my opinion out there now: 20 Critical Security Controls is crap.  I’m sorry John G and Eric C.  Not only is 20 CSC bad from a perspective of controls, metrics, and auditing tests, but if it’s implemented across the Government, it will be the downfall of security programs.  I really believe this.

Now on to the rationale….

Opportunity Costs. I can’t get that phrase out of my head.  And I’m not talking money just yet–I’m talking time.  See, I’m an IT security guy working for a contractor supporting a Government agency–just like 75% of the people out there.  I have a whole bunch of things to do–both in the NIST guidelines and organizational policy.  If you add anything else to the stack without taking anything away,  all you’ve done is to dilute my efforts.  And that’s why I can’t support 20 CSC–they’re an unofficial standard that does not achieve its stated primary goal of reducing the amount of work that I have to do.  I know they wanted to create a parallel standard focusing on technical controls but you have to have one official standard because if it’s not official, I don’t have to do it and it’s not really a standard anymore, it’s it?

Scoping Problems. We really have 2 tiers inside of an agency that we need to look at: the Enterprise and the various components that depend on the Enterprise.  Let’s call them… general support systems and major applications.  Now the problem here is that when you make a catalog of controls, some controls are more applicable to one tier than the other.  With 20 CSC, you run the classic blunder of trying to reinvent the wheel for every small system that comes along.

Threat Capabilities != Controls. And this is maybe the secret why compliance doesn’t work like we think it will.  In a nice theoretical world, it’s a threat-vulnerability-countermeasure coupling and the catalog of controls accounts for all likely threats and vulnerabilities.  Well, it doesn’t work that way:  it’s not a 1-to-1 ratio.  Typical security management frameworks start from a regulatory perspective and work their way down to technical details while what we really want to do is to build controls based on the countermeasures that we need.  So yeah, 20 CSC has the right idea here, the problem is that it’s a set of controls created by people who don’t believe in controls–the authors have the threat and vulnerability piece down and some of the countermeasures but they don’t understand how to translate that into controls to give to implementers and their auditors.  The 20 CSC guys are smart, don’t get me wrong, but I can’t help but get the feeling that they don’t understand how the “rest of the world” is getting their job done out there in the Enterprise.

The Mapping is Weak. There is a traceability matrix in the 20 CSC to map each control back to NIST controls.  It’s really bad, mostly because the context of 800-53 controls doesn’t extend into 20 CSC.  I have serious heartburn with how this is presented inside the agencies because we’re not really doing audits using the 20 CSC, we’re using the mapping of NIST controls with a weird subtext and it’s a “voluntary assessment” not an audit.

Guidelines?!?!?! This is basic stuff.  If it’s something you audit against, it *HAS* to be a standard.  Guidelines are recommendations and can add in more technique and education.  Standards are like hard requirements, they only work if they’re narrowly-scoped, achievable, and testable.  This isn’t specific to 20 CSC, the NIST Risk Management Framework (intended to be a set of guidelines also) suffers from this problem, too.  However, if your intent is to design a technical security and auditing standard, you need to write it like a standard.  While I’m up on a soapbox, for the love of $Diety, quit calling security controls “requirements”.

Auditor Limitations. Let’s face it, how do I get an auditor to add an unmanaged device to the network and know if we’ve detected it or not.  This is a classic mistake in the controls world:  assuming that we have enough people with the correct skillsets who can conduct intrusive technical tests without the collusion of my IT staff.

And the real reason why I dislike the 20 Critical Security Controls:

Introduction of “Audit Requirements”. One of the chief criticisms of the NIST Risk Management Framework is that the controls are not specific enough.  20 CSC falls into this trap of nonspecificity (Controls 7, 8, 9, and 15, I’m talking to you) and is not official guidance–a combination that means that my auditor has just added requirements to my workload simply because of how they interpret the control.  This is very dangerous and why I believe 20 CSC will be the end of IT security as we know it.

In future posts (I had to break this into multiple segments):

• Control-by-control analysis
• What 20 CSC got right (Hey, some of it is good, just not for the reasons that it’s supposed to be good)

SA-2 “Guideline” photo by cliff1066™.

Bookmark to:

Hide Sites

A small presentation Dan Philpott and I put together for Potomac Forum about getting sane social media policy out of your security staff. I also recommend reading something I put out a couple of months ago about Social Media Threats and Web 2.0.

Bookmark to:

Hide Sites

For those of you who didn’t know the real purpose of DojoCon, it was to raise money and awareness for Hackers for Charity. If you like anything that is in this post, go to HFC and make a donation of time, equipment, tech support, and maybe money. If you’ve never heard of HFC because you’re not one of the “InfoSec Cool Kids”, now is your chance–go read about them.

The video of my dojocon presentation. The microphone was off for the first couple of minutes but I look pretty animated.

And then the compliance panel that I tried not to dominate:

And finally, my slides are up on slideshare:

Bookmark to:

Hide Sites

I sat down with Jim Manico a month or so ago when he was in DC and recorded a podcast for the OWASP Podcast.  It’s now live, check it out.

Bookmark to:

Hide Sites

OK, I know you’re shocked…I’m saying something controversial.  But hear me out on this one, I’ll explain.

Now this is my major beef with the way we write SSPs today:  this is all information that is contained in other artifacts that I have to pay people to do cut-and-paste to get it into a SSP template.  As practiced, we seriously have a problem with polyinstantiation of data in various lifecycle artifacts that is cut-and-pasted into an SSP.  Every time you change the upstream document, you create a difference between that document and the SSP.

This is a practice I would like to change, but I can’t do it all by myself.

This is the skeleton outline of an SSP from Special Publication 800-18, the guide to writing an SSP:

1. Information System Name/Title–On the investment/FISMA inventory, the Exhibit 300/53, etc
2. Information System Categorization–usually on a FIPS-199 memorandum
3. Information System Owner–In an assignment memo
4. Authorizing Official–In an assignment memo
5. Other Designated Contacts–In an assignment memo
6. Assignment of Security Responsibility–In assignment memos
7. Information System Operational Status–On the investment/FISMA inventory, the Exhibit 300/53, etc
8. Information System Type–On the investment/FISMA inventory, the Exhibit 300/53, etc
9. General System Description/Purpose–In the design document, Exhibit 300/53
10. System Environment–Common controls not inside the scope of our system
11. System Interconnections/Information Sharing–from Interconnection Security Agreements
12. Related Laws/Regulations/Policies–Should be part of the system categorization but hardly ever is on templates
13. Minimum Security Controls–800-53 controls descriptions which can easily be done in a Requirements Traceability Matrix
14. Information System Security Plan Completion Date–specific to each document
15. Information System Security Plan Approval Date–specific to each document

Now some of this has changed in practice a little bit–# 10 can functionally be replaced with a designation of common controls and hybrid controls.

So my line of thinking is that if we provide a 2-6-page system description with the names of the “guilty parties” and some inventory information, controls-specific Requirements Traceability Matrix, and a System Design Document, then we have the functional equivalent of an SSP.

Why have I declared an InfoSec fatwah against SSPs as currently practiced?

Well, my philosophy for operation is based on some concepts I’ve picked up through the years:

• Why run when you can walk, why walk when you can sit, why sit when you can lay down.  There is a time to spend effort on determining what the security controls are for a project.  You need to have them documented but it’s not cost-effective to be worried about format, which we do probably too much of today.
• Make it easy to do the right thing.  If we polyinstantiate security information, we have made something harder to maintain.  Easier to maintain means that it will get maintained instead of being shelfware.  I would rather have updated and accurate security information than overly verbose and well-polished documents that are inaccurate.
• Security is not a “security guy thing”–most problems are actually a management and project team problem.  My idea uses their SDLC artifacts instead of security-specific versions of artifacts.  My idea puts the project problems back in the project space where it belongs.
• If I have a security engineer who has a finite amount of hours in a day, I have to choose what they spend their time on.  If it’s a matter of vulnerability mitigation, patching, etc, or correcting SSP grammar, I know what I want him to do.  Then again, I’m still an infantryman deep down inside and I realize I have biases against flowery writing.

Criticisms to not writing a dedicated SSP document:

“My auditors are used to seeing the information in the same format at someplace they worked previously”. Believe it or not, I hear this quite a bit.  My response is along the lines of the fact that if you make your standard be what I’m suggesting for a security plan, then you’ve met all of the FISMA and 800-53 requirements and my personal requirement to “don’t do stupid stuff if you can help it”.

“My auditors will grill me to death if they have to page back and forth between several documents”.  This one also I’ve heard.  There are a couple of ways to deal with this.  One way to deal with this is that in your 800-53 Requirements Traceability Matrix you reference the source document.  Most auditors at this point bring up that you need to reference the official name, date of publication, and specific page/section of the reference and I think they need to get a life because they’ve taken us back to the maintainability problem.

“This is all too new-school and I can’t get over it”. Then you are a dinosaur and your kind deserves extinction.  =)

.

This blog post is for grecs at novainfosecportal.com who perked up instantly when I mentioned the concept months ago.  Finally got around to putting the text somewhere.

How to Plan the Perfect Dinner Party photo by kevindooley.

Bookmark to:

Hide Sites