Security Assessment Economics

Posted June 12th, 2008 by

I’ve spent a couple of days traveling around to agencies to teach.  It was fun but tiring, and the best part of it is that since I’m not teaching pure doctrine, I can include the “here’s how it works in real life” parts and some of the BSOFH parts–what I refer to as the “security management heretic thoughts”.

Some basic statements, the rest of this post will explain:

  • C&A is a commodity market
  • Security controls assessment is a commodity market
  • PCI assessment is a commodity market
  • Most MSSP (or rather, Security Device Management Service Providers) services are commodity markets

Now my boss said the first one to me about 4 months ago and it really needed some time for me to grasp the implications.  What we mean by “commodity market” is that since there isn’t really much of a difference between vendors, the vendors have to compete on having the lower price.

Now what the smart people will try to do is to take the commodity service and try to make it more of a boutique service by increasing the value.  Problem is that it only works if the customers play along and figure out how your service is different–usually what happens is you lose in the market simply because now you’re “too expensive”.

Luxury, Boutique, Commodity

Where Boutique Sits by miss_rogue.

Since the security assessment world is a services business, the only way to compete in a commodity market is to pay your people less and try to charge more. But oh yeah, we compete on price, so that only leaves the paychecks as the way to keep the margin up.

Some ways that vendors will try to keep the assessment costs down:

  • Hire cheaper people (yes, paper CISSPs)
  • Try to reduce the engegement to a formula/methodlogy (ack, a checklist)
  • It’s all about billability:  what percentage of your people’s time is not billable to clients? 
  • Put people on assessments who have tangential skills just to keep them billable
  • Use Cost-Plus-Margin or Time-Plus-Materials so that you can work more hours
  • Use Firm-Fixed-Price contracts with highly reduced services ($150 PCI assessments)

Now inside Government contracting, there’s a fact that’s not known outside of the beltway:  your margins are fixed by the Government.  In other words, they only allow you to have around a 13-15% margin.  The way to make money is that the pie is a much bigger pie, even though you only get a small piece of it.  And yes, they do look at your accounting records and yes, there are loopholes, but for the most part, you can only collect this little margin.  If you stop and think about it, the Government almost forces the majority of its contractors into a commodity market.

Then we wonder why C&A engagements go so haywire…

The problem with commodity markets and vulnerability/risk/pen-test assessments is that your results, and by extension your ability to secure your data, are only as good as the skills and creativity of the people that the vendor sends.  Sounds like a problem?  It is.

So knowing this, how can you as the client get the most out of your service providers? This is a quick list:

  • Every year (or every other), get an assessment from somebody who has a good reputation for being thorough (ie, a boutique)
  • Be willing to pay more for services than the bottom of the market but be sure that you get quality people to go along with it, otherwise you’ve just added to the vendor’s margin with no real improvements to yourself
  • Get assessments from multiple vendors across the span of a year or two–more eyes means different checklists
  • Provide the assessors with your own checklists so you can steer them (tip from Dave Mortman)
  • Self-identify vulnerabilities when appropriate (especially with vulnerabilities from previous assessments)
  • Typical contracting fixes such as scope management, reviewing resumes of key personnel, etc
  • Get lucky when the vendor hires really good people who don’t know how much they’re really worth (that was me 5 years ago)
  • More than I’m sure will end up in the comments to this post  =)

And the final technique is that it’s all about what you do with the assessment results.  If you feed them into a mitigation plan (goviespeak: POA&M) and improve your security, it’s a win.

Similar Posts:

Posted in Outsourcing, Rants, Risk Management, The Guerilla CISO | 6 Comments »

6 Responses

  1.  Jeremy Says:

    I think that your basic statements follow some of your previous posts very well.

    It is a commodity market because we have been reduced to checklists that “anyone” can fill out. It is again the compliance vs. risk management argument.

    The problem lies in, as you pointed out, the “paper CISSPs”. I am not expecting everyone to be a techincal expert in everything. There are a lot of places where I fail. But, I am somewhat shocked at the lack of technical background in some CISSPs I have met.

    I like checklists in that they provide a “consistant” baseline to compare systems on. However, when the analysts don’t understand what they are grading, I am afraid that the quality suffers. Even though you are doing a quantitative checklist, it is still someone’s opinon.

  2.  Interesting Information Security Bits for June 13th, 2008 « Infosec Ramblings Says:

    […] Guerilla CISO has some observations on security services as commodities and the implications of such how those services are provided. Something to think […]

  3.  Graydon McKee Says:

    I think that you’re dead on. The problem is that the customer doesn’t know how to discern between a quality provider and a “checklist” provider.

    I think that is the $64,000 question. Certs only prove that you’ve met the minimum knowledge level to pass the test. How do you judge a company over an individual? This is a key question for me as I’m in the process of starting my own company to provide these services.

  4.  NIST’S FISMA Pase II–Who Certifies Those who Certify the Certifiers? | The Guerilla CISO Says:

    […] services are a commodity market:  I covered this last week.  This is pivotal, remember it for […]

  5.  Evolution of Penetration Testing: Part 2 | The Guerilla CISO Says:

    […] blog I outlined the fact penetration testing evolved from a grey-art practiced by hackers into a more formal process.  This evolution has created a bifurcation within “boutique” penetration test service […]

  6.  Tweets that mention Security Assessment Economics | The Guerilla CISO -- Says:

    […] This post was mentioned on Twitter by Christian Martorella and Rafael Alfaro March, Jose Selvi. Jose Selvi said: Si quieres algo bueno, paga bien, y sino no te quejes RT @laramies: Security assessment economics interesting post. […]

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

Visitor Geolocationing Widget: