The “Off The Record” Track

Posted November 21st, 2011 by

So while I was at some conferences over the past couple of months, I had an awesome idea while sitting in a panel about data breaches, especially notification. While streaming conferences is pretty awesome for most content, I keep thinking that we need that as an industry we need the exact opposite: a track of the conference that is completely off-the-record.

Here in DC when we do smaller training sessions, we invoke the Chatham House Rule.  That is, the discussion is for non-attribution.  There are several reasons behind this:

  • You don’t have to worry (too much, anyway) about vendors in attendance selling you something
  • It won’t end up in the press
  • It gets real information to people instead of things that are “fit for public consumption”

My local area has a hackers association (No linkie, if you have minimal skill you can find it) that meets to talk about mostly technical stuff and what folks are working on.  I find that more and more often when I do a talk there I do it “Off the Record” for a wide variety of reasons:

  • I don’t want the attackers to get more effective
  • I have half-baked ideas where I want/need feedback on if they are completely off-base
  • The subject matter is in a legal gray-area and I’m not a lawyer
  • I talk “on the record” all day every day about the same things
  • I can “test-drive” presentation material to see how it works
  • I can show nuts and bolts

So, the point of all this is that maybe we need to start having more frank discussions about what the bad guys are doing “in the wild” if we want to stop them, and that involves talking with peers from other companies inside the same industry to see what they are getting hit with.

Chatham House Rule

Chatham House Rule photo by markhillary.

Similar Posts:

Posted in Public Policy, Speaking, What Doesn't Work, What Works | 3 Comments »

DojoCon DDoS Video

Posted December 16th, 2010 by

My DDoS presentation at DojoCon on Sunday.  A big thanks to Marcus J Carey for organizing the con and Adrian Crenshaw for doing the recording.

Michael Smith, @rybolov DDoS from Adrian Crenshaw on Vimeo.

Similar Posts:

Posted in Cyberwar, Speaking, Technical, What Doesn't Work, What Works | 2 Comments »

Engagement Economics and Security Assessments

Posted September 29th, 2010 by

Ah yes, I’ve explained this about a hundred times this week (at that thing that I can’t blog about, but @McKeay @MikD and @Sawaba were there so fill in the gaps), thought I should get this down somewhere.

the 3 factors that determine how much money you will make (or lose) in a consulting practice:

  • Bill Rate: how much do you charge your customers.  This is pretty familiar to most folks.
  • Utilization: what percentage of your employees’ time is spent being billable.  The trick here is if you can get them to work 50 hours/week because then they’re at 125% utilization and suspiciously close to “uncompensated overtime”, a concept I’ll maybe explain in the future.
  • Leverage: the ratio of bosses to worker bees.  More experienced people are more expensive to have as employees.  Usually a company loses money on these folks because the bill rate is less than what they are paid.  Conversely, the biggest margin is on work done by junior folks.  A highly leveraged ratio is 1:25, a lowly leveraged ratio is 1:5 or even less.

Site Assessment photo by punkin3.14.

And then we have the security assessments business and security consulting in general.  Let’s face it, security assessments are a commodity market.  What this means is that since most competitors in the assessment space charge the same amount (or at least relatively close to each other), this means some things about the profitability of an assessment engagement:

  • Assuming a Firm Fixed Price for the engagement, the Effective Bill Rate is inversely proportionate to the amount of hours you spend on the project.  IE, $30K/60 hours=$500/hour and 30K/240 hours = $125/hour.  I know this is a shocker, but the less amount of time you spend on an assessment, the bigger your margin but you would also expect the quality to suffer.
  • Highly leveraged engagements let you keep margin but over time the quality suffers.  1:25 is incredibly lousy for quality but awesome for profit.  If you start looking at security assessment teams, they’re usually 1:4 or 1:5 which means that the assessment vendor is getting squeezed on margin.
  • Keeping your people engaged as much as possible gives you that extra bit of margin.  Of course, if they’re spending 100% of their time on the road, they’ll get burned out really quickly.  This is not good for both staff longevity (and subsequent recruiting costs) and for work quality.

Now for the questions that this raises for me:

  • Is there a 2-tier market where there are ninjas (expensive, high quality) and farmers (commodity prices, OK quality)?
  • How do we keep audit/assessment quality up despite economic pressure?  IE, how do we create the conditions where the ninja business model is viable?
  • Are we putting too much trust in our auditors/assessors for what we can reasonably expect them to perform successfully?
  • How can any information security framework focused solely on audit/assessment survive past 5 years? (5-10 years is the SWAG time on how long it takes a technology to go from “nobody’s done this before” to “we have a tool to automate most of it”)
  • What’s the alternative?

Similar Posts:

Posted in Rants, What Doesn't Work | 3 Comments »

Split-Horizon Assessments and the Oversight Effect

Posted July 7th, 2010 by

Going Off the Deep End

So I was thinking the other day (this is the part where people who know me in person usually go “oh cr*p”), partially spurred by a conversation I had with @csoandy and @secbarbie a couple of months ago.  I’ll get the idea out there: as an industry we need to embrace the concept of split-horizon assessments.

Two Purposes for Assessments

Because this is an insane approach that I’m just feeling out, let me go on a solo riff and explain what I’m talking about.  You see, I have two distinct purposes for getting a security assessment, both of which are in contention with each other:

  • I want to fix my security by asking for money to fix the things that need attention.  When I get an assessment for this purpose, enumeration of my badness/suckness is good.  If I have a set of results that say that everything is great, then there’s no need for me to be given any more resources (time, money, people, gear).  Short-term, I’m fine, but what about my infrastructure-type long-term projects?  The net effect of a highly-scored annual assessment just might kill my program in 2 years as my funding and people are shifted elsewhere, especially in a .
  • I want to keep my job and help my {company|agency|group} stay out of trouble by showing my zero-defects face and by demonstrating my due-diligence in protecting what has been given to me.  While the assessor has helped me short-term by identifying my problems and being a total hardass, if I’m not around in 6 months to adopt the recommendations into my security program, has the assessor actually helped me?

And this is the dilemma for just about every security manager out there.  One of the strategies is to alternate assessment types, but then your management wonder just what the heck it is you’re doing because you’re on top one year, then on the bottom the next.

Split Rock Lighthouse and Horizon photo by puliarf.

Assessor Window-Shopping

Now for the dirty little secret of the testing business:  there are really good testers who are the ninjas of the InfoSec world and there are really bad testers who don’t even validate their unlicensed Nessus scan.  I know, you’re shocked and it’s so blindingly obvious that Bruce Schneier will blog it 3 years from now.  =)

But there’s the part that you didn’t know:  security managers pick their assessor depending on the political mood inside their organization.  This is nowhere near a science, from what I’ve seen it involves a lot of navel-gazing on the part of the security team to see which is the lesser evil: having everybody think you’re incompetent or never getting anything new ever again?

Building a Better Rat Race

In order to accomplish both of the goals that I’ve listed, what I really need is a split-horizon assessment.  In other words, I need 2 reports from one assessment with different views for different audiences.  I know this sounds highly cynical, but it’s something we’ve been doing for some time now but just informally.  Might as well make it formal.

So are you sold on this concept yet?  In true form, I have an idea on how to get to a world of split-horizon assessments.  You can take any catalog of controls and divide it into “gotta have it” and “nice to have” (I almost divide these along the lines of “vulnerability mitigation” and “sustainable security program” or the “CISO” and “OMB and Congress”) buckets.  Then in your compliance assessment standard, require 2 reports for each assessment.  One is reported to the regulating authority and the other stays with the organization.

Indecision Strikes

I don’t know if I’ve solved the problemspace or not, but I’m looking for feedback “from the Peanut Gallery” so leave some comments.

Similar Posts:

Posted in Rants, What Doesn't Work, What Works | 7 Comments »

A Funnier Thing Happened on the WAY to Capitol Hill

Posted June 15th, 2010 by

Since I never get to see Vlad the Impaler enough in real life I was pleased to see his recent blog post, “Machines Don’t Cause Risk, People Do!”. It reminded me of fact that security professionals must have so-called people skills as well as a keen insight into the dynamics and group psychology of organizations in order to be effective.

As is true for technological solutions, security controls and security policy must also be subject to the concepts and process found in life-cycle methodologies. As security professionals we must be constantly aware of these cycles as in some cases this means that controls and policies can outlive their usefulness. In other cases it means that security policies, concepts, and policies can evolve or mutate until they are no longer viable or meaningful.

It is the later phenomena that that caught my attention recently. But, first let me set the stage…

A Tragic History

Back in 1983 the American people were made aware of the concept of a truck bomb in a dramatic and tragic fashion. In late October of that year, truck bombers attacked the compounds housing U.S. and French peacekeepers in Lebanon. The loss of life was shocking.

In the aftermath of this tragedy there was a great deal of political finger pointing. Notably, security professionals had expressed concerns about the vulnerability of the deployment and had made several recommendations to improve the security of the facility. Some of the recommendations were followed, others that would have greatly mitigated the against the damage and loss of life in the subsequent attack were not implemented. In addition, security professionals were also asked to rise above all of the politics and examine the situation from a, “lessons-learned” perspective and develop generally applicable counter-measures. One obvious and immediate response was the introduction of the bollards or jersey barriers around public and government buildings. While experts agreed that this wasn’t a complete solution to the problem of the vehicular bomb, it was and still is seen as a useful and essential tool.

Two criticisms to the use of these physical barriers were quickly voiced. The first criticism focused on the aesthetics of these barriers. Critics correctly pointed out that the barriers that were initially introduced were ugly and made public building buildings and spaces protected by these barriers take on an unfriendly fortress-like appearance. After a time the response to this was the introduction of barriers that were masqueraded as sculpture, large planter boxes and even seats or benches.

The second criticism focused on the fact that many public building and spaces were constructed in such a fashion such that it was difficult, expensive, and in some cases even impossible to effectively employ these barriers. A common problem noted was that building was often constructed with little or no “set-back” between the building and streets. This meant that there is no meaningful way in which to erect barriers at a sufficient distance from the building in question to afford it any meaningful protection.

Within the limits of always constrained budgets, the Federal government began erecting vehicular barriers all over the country and even overseas. The government also began a program that developed a risk and vulnerability assessment or classification of all Federal facilities and buildings.

History Repeats Itself With a New Twist

Ten years later, the US was horrified again by the bombing of Federal Building in Oklahoma City. While the bombing and loss of life was a terrible tragedy in the truest sense of the word, the similarities of this incident to the 1983 incident made it all that much more painful. The fact that the Oklahoma City tragedy took place domestically and resulted from entirely domestic terrorist plotters made the situation even more sobering.

Even worse, because the above mentioned security assessment classified the Oklahoma City Federal Building as being a relatively low risk facility. There were two significant consequences to this security assessment/classification. The first was that the use of extensive anti-vehicle barriers or bollards were seen as being unnecessary. The second was that the building was seen as safe enough that a day care facility was approved for the building. This decision added an additional element of heartbreak to the general feeling of horror and grief in response to the bombing.

A Thoughtful Response

In the aftermath of this terrible act the Federal government develop a rather extensive set of building specification that were required in all new construction. When implemented, these specifications greatly increase a building ability to resist a similar attack. Moreover, this risk-based specification focuses considerable attention on reducing the risks to the people in the building. For example, protective films are required for all windows, thus reducing the risks from flying fragmented glass.

Because of the extended thought that went into this specification, many of the technologies and approached embraced in the specification are also available as affordable retrofits to existing building. This is especially useful in the case of leased building or office space.

Having had an opportunity to work with these codes and specification, my impression is that there is a good deal of sound thinking behind these measures. Moreover, these specifications are constantly reviewed and updated taking into account the latest threats and the technical developments.

Security Meets the Street

A few weeks ago I was walking down Pennsylvania Avenue, in Washington D.C. I was a beautiful day and I was just a few blocks from the White House. I was a little surprised when I saw one of bollards that I mentioned earlier. The bollard itself isn’t all that surprising; they are a pretty common site around the nations’ capital. The fact that this particular barrier was masquerading as a planter box for a small tree was also not all that unusual. However, the barrier was damaged.. At a casual glace a damaged bollard also isn’t all that unusual a sight either. But, with a quick glance at this bollard something in the back of my mind whispered that there was something odd about this barrier. I looked at the damaged area and noticed that the bollard was filled with Styrofoam. That seemed odd enough to catch my attention and motivated me to investigate a bit further.

The first thing I did was to take a quick snap-shot of the bollard (see below). I can’t say it’s likely that I will ever will a Pulitzer Prize for photo journalism, but if you examine my snapshot closely you can clearly identify the Styrofoam grains in the damaged section photographed. I also had a bit of luck. Just as I was looking at the barrier one the incredibility efficient and effective D.C. Parking Enforcement Offices just happened by plying their trade. I asked them I they were aware of what happened to the barrier in question. I was in luck; the officer was an eye-witness to a minor fender bender in which the bollard was damaged. I pointed out the foam filling and asked if what the point of the foam was. She informed me that the barriers had to be moved all the time. Older planter boxes were constructed from solid poured cement or aggregate but, they were heavy and difficult to move. So, in response to this problem, they introduced the “improved” lighter weight barriers. I pointed out that it didn’t seem to be very durable and therefore didn’t seem to be a very effective barrier to a vehicle driven by a determined individual. She laughed and shared with me they were so fragile that the crews that moved them often damaged or destroyed them just by moving them.

Concrete and Styrofoam

Styrofoam in Concrete Barrier photo by Ian.

I guess at that point my incredulous look was obvious on my face and the officer responded to my unasked question by say, “I just write tickets; have a nice day!”


Perhaps I’m over-reacting to what I saw and heard. However, this seems to be a good example of how an essential security control can be compromised for reasons completely unrelated to security. In this case, it isn’t clear what the role of the security professionals involved in this process was. They could have fought the weakening of this security control to the limits of their ability. It is also possible that the warning of the security-types were lost in the shuffle between the various Federal and city jurisdictions involved in this situation. Convenience and practicality are often the enemy of security policy and security implementation. On the surface of it, this seems to be a good case study making that point.

This is perhaps an example of one of the most difficult and frustrating aspects of the responsibilities of the security community — especially for security leaders. We must hold the line and do the right thing. We will never be thanked for it. And, we constantly risk having our jobs or reputations put at risk for doing the right thing and fighting the good fight. But, it is important to remember that the consequences of ignoring this responsibility are even larger and potentially graver that job security. I know that Vlad is a hard-nosed security professional who will not compromise. If he is over-ruled, and that happens, he still sleeps well at night.

Similar Posts:

Posted in Risk Management, What Doesn't Work, What Works | 2 Comments »

How to Not Let FISMA Become a Paperwork Exercise

Posted June 7th, 2010 by

OK, since everybody seems to think that FISMA is some evil thing that needs reform, this is the version of events on “Planet Rybolov”:

Goals to surviving FISMA, based on all the criticisms I’ve read:

  • Reduce paperwork requirements. Yes, some is needed.  Most is not.
  • Reduce cost. There is much repetition in what we’re doing now, it borders on fraud, waste, and abuse.
  • Increase technical effectiveness. IE, get from the procedural and managerial tasks and get down into the technical parts of security.

“Uphold our Values-Based Compliance Culture photo by kafka4prez.

So now, how do you keep from letting FISMA cripple you or turn into death-by-compliance:

  • Prioritize. 25% of your controls need to not fail 100% of the time.  These are the ones that you test in-depth and more frequently.  Honestly, how often does your risk assessment policy get updated v/s your patch management?  Believe it or not, this is in SP 800-53R3 if you interpret it in the correct context.  More importantly, do not let your auditors dictate your priorities.
  • Use common controls and shared infrastructure. Explicitly tell your system owners and ISSOs what you are providing as the agency CISO and/or the GSS that they are riding on.  As much as I hate meetings, if you own a General Support System (GSS), infrastructure (LAN/WAN, AD Forest, etc), or common controls (agency-wide policy, budget, Security Operations Center, etc), you have a fiduciary, legal, and moral obligation to get together with your constituency (the people who rely on the security you provide) and explain what it is you provide and allow them to tell you what additional support they need.
  • Share Assessment Results. I’m talking about results from service providers with other agencies and systems.  We’re overtesting on the high-level stuff that doesn’t change and not on the detailed stuff that does change.  This is the nature of security assessments in that you start at the top and work your way down into the details, only most assessments don’t get down into the details because they’re busy reworking the top-level stuff over and over again.  Many years ago as a contractor managing infrastructure that multiple agencies used, it was unbelievably hard to get one agency to allow me to share security documents and assessment results with other agencies.  Shared assessment results mean that you can cut through the repetitious nature of what you’re doing and progressively get deeper into the technical, frequently-changing security aspects.
  • Simplify the Paperwork. Yes, you still need to document what you’re doing, but the days of free-text prose and being graded on grammar and punctuation need to be over.  Do the controls section of System Security Plans as a Requirement Traceability Matrix.  More important than that, you need to go by-control by-component.  If you are hiring contractors and their job is to do copypasta directly from NIST documents and change the pronouns and tenses, you’re doing it wrong.  Don’t stand for that in your security policy or anything else that you do.
  • Automate Wherever Possible. Note that the controls that change frequently and that need to not fail usually fit into this group.  It’s one of those “Things that make Rybolov go ‘Hmmmm'”.  Technology and automation provide both the problem and the solution.  Also see my first point up above.
  • Fire 50% of Your Security Staff. Yes, I’m serious.  Those people you didn’t need anyway, primarily because they’re violating all the points I’ve made so far.  More importantly, 25 clueless people can mess things up faster than 5 clueful people can fix them, and that’s a problem for me.  Note that this does not apply to @csoandy, his headcount is A-OK.

The incredible thing to me is that this stuff is already there.  NIST writes “hooks” into their Special Publications to allow the smart people the room to do all these things.

And now the part where I hop up on my soapbox:  reforming FISMA by new legislation will not make any achievements above and beyond what we have today (with the exception of creating a CISO-esque position for the Exective Branch) because of the nature of audit and compliance.  In a public policy sense, the more items you have in legislation, the more the audit burden increases and the amount of repetition increases, and the amount of nonsense controls (ie, AntiVirus for Linux servers) increases.  Be careful what you ask for, you just might get it.

Similar Posts:

Posted in FISMA, NIST, Rants, Risk Management, What Doesn't Work, What Works | 2 Comments »

« Previous Entries

Visitor Geolocationing Widget: