Evolution of Penetration Testing: Part 1

October 13th, 2008 by ian99

If you're new here and would like to see more of what I'm saying, you may want to subscribe to my RSS feed (I can even email my blog posts to you when I publish a new one) or have a look at my papers and presentations page for downloads of stuff that you can share or "borrow heavily from". You also might find my guidelines for posting comments interesting, especially if you're a government employee. If you want to see me blog about anything in particular, drop me a private email on how you think I'm completely full of myself, extend me an invitation to speak at your next security meeting/event, or just to ship a huge bag of money in my direction, you can do that through my contact page. Thanks for visiting and happy hacking!

Penetration testing is a controversial topic with an interesting history. It is made all that much more controversial and perplexing because of an common disconnect between the service provider and the consumer.

Penetration started as a grey-art that was often practiced/delivered in an unstructured and undisciplined manner by reformed or semi-reformed hackers. Penetration testers used their own techniques and either their own home-grown tools or tools borrowed or traded with close associates. There was little reproducibility or consistency of results or reporting. As a result, the services were hard to integrate into a security program.

As the art evolved it became more structure and disciplined and tools, techniques, and reporting became more standardized. This evolution was driven by papers, articles, technical notes that were both formally published and informally distributed. In the end, a standardized methodology emerged that was largely based on the disciplined approach used by the most successful hackers.

Hakker Kitteh photo by blmurch.

At about the same time open-source, government and commercial tools began to emerge that automated many of the steps of the standardized methodology. These tools had two divergent impacts on the art of penetration testing. As these tools were refined and constantly improved they reinforced the standard methodology, provided more consistent and reproducible results and improved and standardized penetration reporting. All of this made penetration testing easier for the consumer to absorb and integrate into security programs. As a result, regulations and security protocols emerged that required penetration and security assessments. Nmap and Nessus are excellent examples of the kind of tools that help shape and push this evolution. And, because of their utility they are still indispensable tools today.

However, Nessus also helped to automate both data collection and analysis, it has lowered the bar for the skills and experience needed to conduct portions of the penetration testing methodology. This lowered the cost of penetration testing and made them much more broadly available. Thus, giving rise to so-called “boutique firms.” The problem with penetration testing “boutique firms” is that they fall into two broad categories; specialized highly professional firms led by experienced and technical security professionals who can translate automated tool output into root-cause analysis of vulnerabilities, and security program flaws. The second category of firm consists of opportunist firms with just enough knowledge to run automated tools and cut and paste the tool output into client reports. The later firms are some times called “tool-firms” and their employees “tool-boys.”

The later flourish for two reasons. The first is that they can offer their services at rock bottom prices. The second reason is that security organizations are often so ill-informed of the intricacies of the penetration testing process that can’t make a meaningful distinction between the professional firms and the tool-boys except on the basis of costs.

Posted in Rants, Technical | 2 Comments »

Et Tu, TIC?

October 7th, 2008 by rybolov

Let’s talk about TIC today, dear readers, for I smell a conspiracy theory brewing.

For those of you who missed the quick brief, TIC is short for “Trusted Internet Connections” and is an architecture model/mandate/$foo to take all of the Internet connections in the Government (srsly, nobody knows how many of them really exist, but it’s somewhere in the 2,000-10,000 range) and consolidate them into 50.  These connections will then be monitored by DHS’s Einstein program.

No, Not That Kind of TIC photo by m.prinke.

Bringing you all up to date, you’ll need to do some homework:

Now having read all of this, some things become fairly obvious:

  • If you have the following people needing connections:
    • 24 agencies, plus
    • DoD with 2 points of presence, plus
    • Intelligence agencies with a handful of Internet connections, means that:
  • That basically, everybody gets one Internet connection.  This is not good, it’s all single point-of-DOS.
  • Agencies have been designated as Internet providers for other agencies.  Sounds like LoB in action.
  • Given the amount of traffic going through the TIC access points, it most likely is going to take a significant amount of hardware to monitor all these connections–maybe you saved 50% of the monitoring hardware by reducing the footprint, but it’s still hardware-intensive.
  • TIC is closely tied with the Networx contract.
  • In order to share Internet connections, there needs to be a network core between all of the agencies so that an agency without a TIC access point can route through multiple TIC service provider agencies.

And this is where my conspiracy theory comes in:  TIC is more about making a grand unified Government network than it is monitoring events–Einstein is just an intermediate goal.   If you think about it, this is where the Government is headed.

We were headed this way back in ought-two with a wonderful name: GovNet.  To be honest, the groundwork wasn’t there and the idea was way ahead of its time and died a horrible death, but it’s gradually starting to happen, thanks to TIC, FDCC, and Einstein. 

More fun links:

If you want to get a reaction out of the OMB folks, mention GovNet and watch them backpedal and cringe,–I think the pain factor was very high for them on GovNet. So I think that we should, as a cadre of information security folks, start calling TIC what it really is:  Govnet 2.0!  =)

Posted in Technical | 1 Comment »

TIC: Made for LOLCATS

October 2nd, 2008 by rybolov

Well, other than the fact that I think TIC isn’t about reducing the attack footprint of the Government (more to follow on this), it makes a fun compliance pinata to whack at.

funny pictures

Posted in IKANHAZFIZMA | No Comments »

Workin’ for the ‘Counters: an Analysis of my Love-Hate Relationship with the CPAs

September 30th, 2008 by rybolov

No big surprise by now, I work for an accounting firm.  Oh, what’s that?  Oh yes, that’s right, it’s a consulting firm with a high percentage of accountants, including a plethora of CPAs.  “Accounting firm” is so 1950s-ish. =)

It’s my secret theory (well, not so much of a secret now, just between the Internet and me) that the primary problem we have in information security is that as a field we have borrowed heavily from public accounting.  The only problem is that public accounting is different from what we do.

Goals for public accounting run something like this:

  • Eliminate fraud through oversight
  • Protect the company’s money from rogue agents
  • Protect the shareholders of public companies
  • Ensure accountability of actions

Accounting for Mere Mortals Such as Security Folk

Accounting for Non-Accountants photo by happyeclair.

As a result of their goals, accountants have an interesting set of values:

  • Signatures are sacred
  • Separation of duties is sacrosanct
  • Auditing is designed to act as a deterrent to fraud
  • “Professional Skepticism” is a much-valued trait
  • Zero-Defects is a good condition

In other words, accountants live in a panopticon of tranparency, the concept being that through oversight and transparency, people will not become evildoers and those that do will be caught.  Pretty simple idea, makes me think about IDS in an entirely new light.

Words that accountants use that mean something entirely different from the way you or I use them:

  • Fraud, Waste, and Abuse: They’re talking about spending money, I’m usually talking about people doing something ethically wrong.
  • Investigation: They’re looking at the numbers to see how a particular number was created.  Me, I bring the nice people with guns when I do an investigation.
  • Incident: Their version is what I would call an event.  When I call something an incident, we’re headed towards an investigation.
  • Security test and evaluation: To them, it’s a compliance audit.  To me, it’s determining the frequency that the system will fail and if we have a way to fix it once it does.  Remember this, it’s a critical difference.
  • Control: I think their version has something to do with having oversight and separation of duties.  Me, when I see this word, I think “countermeasure to a specific threat and vulnerability”.
  • Audit: An activity designed to prove that fraud has not happened.  Usually we don’t use the word unless we absolutely have to.
  • Technical: They’re talking about the highly-detailed accounting rules.  I’m talking about if you know how to build your own server and OS using lumps of raw silicon and a soldering iron.
  • Checklist: They’re talking about a sacred list that condenses all the rules into an easily-auditable format.  Me, I’m thinking that a checklist is something that will fail because my threats and their capabilities don’t fit into nice little lists.
  • Forensics: Their version is what I would call “research to find out where the money went to” and involves looking at a bunch of numbers.  My version has something to do with logs, memory dumps, and hard drive images.
  • Risk Management: This has something to do with higher interest rates for high-risk loans.  For me, it’s looking for countermeasures and knowing what things to skimp on even though the catalog of controls says you have to have it.

In short, pretty much anything they could say about our line of work has a different meaning.  This is why I believe it’s a problem if we adopt too much of their methodology and management models because they are doing similar activities to what security people do, only for different purposes.

In order to understand the mentality that we’re working with, let’s give you a couple of scenarios:

After-Work Optional Training Session: The accountants not only make you put your name on the attendance roster but you have to sign it as well.  Are they worried that you’re committing fraud by showing up at training that you were not supposed to, so they need some sort of signature nonrepudiation to prove that you were there?  No!  They just make you sign it because they believe in the power of the signature and that’s just how they do things, no matter how trivial.

The Role of Security: To an accountant, the role of security in an organization is to reduce fraud by “hack-proof” configurations and monitoring.  This is a problem in that since security is economics, we’re somehow subordinate to the finance people.

Let’s look at the world of the typical security practitioner:

  • The guidance that security professionals have is very contradictory, missing, or non-relevant.
  • Really what we do comes down to risk management, which means that sometimes it makes more sense to break the rules (even though there is a rule that says break the rules, which should freak your brain out by now if you’re an accountant).
  • We have a constantly changing environment that rules cannot keep up with.

Now this whole blog post, although rambling on about accountants, is aimed at getting a message across.  In the US Federal Government, we use a process called certification and accreditation (C&A).  The certification part is pretty easy to understand–it’s like compliance, do you have it and does it work.  CPAs will readily understand that as a controls assessment.  That’s very much a transferable concept.

But in accreditation, you give the risks to a senior manager/executive and they accept the risks associated with operating the system.  The CPA’s zero-defects world comes through and they lie on the ground doing the cockroach.  Their skills aren’t transferable when dealing with risk management, only compliance with a set of rules.

Once again, the problem with security in Government is that it’s cultural.

And don’t get me wrong, I like accountants and they do what I do not have neither the skills nor the desire to do.  I just think that there aren’t as many transferable skills between our jobs as there might seem on the surface.

Posted in Odds-n-Sods, Rants | 2 Comments »

Keeping The Lights On: Cybersecurity Law for the Electric Grid

September 23rd, 2008 by DanPhilpott

Ever wondered if your electricity supply was safe from computer attack? Congress wondered that too. So they asked the Federal Energy Regulatory Commission (FERC) to find out. The answers they received in October of 2007 were not encouraging.

After 9/11 there was concern about the safety of the Bulk Power Supply (BPS). The President’s Commission on Critical Infrastructure Protection released a report which was explicit about the dangers faced. A frightening example of these dangers was demonstrated by the Aurora vulnerability, essentially a software hack that made a generator crash hard. When faced with this example industry moved to mitigate the problem with some prodding from Department of Homeland Security (DHS), Nuclear Regulatory Commission (NRC) and FERC. The Nuclear Sector, which is regulated by NRC, issued a requirement to address the problem. The Electric Sector was issued a recommendation to address the problem by the Electric Sector Information Sharing and Analysis Center (ES-ISAC). Guess which industry has moved forward with successful mitigation efforts and which has not. FERC reported back on these findings in October of 2007.

Fast forward to now. On September 11th the Bulk Power System Protection Act (BPSPA) of 2008 (PDF link) was put forward by Rep. Rick Boucher (D-VA), chairman of the House Subcommittee on Energy and Air Quality. In addition to the September 11th hearing on the BPSPA a closed door hearing was expected to be conducted the following week. The goal of this legislation is to expand the emergency power of FERC to regulate cybersecurity for the BPS. The act itself does not appear to be strongly opposed by the energy industry but, as always, the devil is in the details.

Diablo Canyon Nuclear Power Plant photo by emdot.

The draft legislation is disputed on three major points; whether to include national security threats, disclosure of threat information and a sunset provision.

FERC recommends wording that would make explicit the requirement to address national security threats. This seems an implicit and reasonable expectation that the people of the United States would have of the agency regulating the BPS but the Energy Sector considers this too expansive a role. They argue that it might cause expensive requirements to be issued such as stockpiling fuel.

The disclosure of threat information is a sore point. Here you can understand the pain of the industry in dealing with government intelligence agencies who would like to keep details of a threat spare to preserve the source of that information. Unfortunately the government must preserve their sources while providing enough information for the industry to react.

Both FERC and the Energy Sector agree on the idea of a sunset provision. The sunset provision in this case stipulates that so long as an order is implemented as a standard it should terminate one year after issuance unless renewed by the President or the Secretary of Energy. The issue is whether this sunset will include the orders to address existing problems (such as the Aurora vulnerability) in addition to orders issued for future vulnerabilities. FERC recommends that only future orders should be sunsetted while the Energy Sector recommends both current and future orders should be sunsetted.

One element which is not adequately addressed in this legislation is how FERC will build the capability to assess and manage cybersecurity issues for the BPS. What should be in place is a bipartite separation of duties between FERC and NIST similar to what is in place with the dual OMB/NIST FISMA roles. FERC would oversee the security while NIST would provide technical guidance on what security should be put in place. FERC does not have the experience in security frameworks or in depth expertise in SCADA security which is required for a cybersecurity initiative of this magnitude.

It is worth noting that Energy Policy Act of 2005 (PDF link) established a process through which the North American Electric Reliability Corporation’s (NERC) was authorized to enforce cybersecurity in the Energy Sector. NERC had gone so far as to create Critical Infrastructure Protection (CIP) standards to include with their Reliability Standards and had present them to FERC for approval by late 2007.

A review of the NERC CIP standards (CIP-001 through CIP-009) does not inspire confidence in NERC’s cybersecurity capabilities. I will discuss the shortcomings of this guidance in a subsequent post.

Posted in What Doesn't Work | 2 Comments »

Super Secret Security Control You Were Never Meant To See

September 4th, 2008 by DanPhilpott

This super secret security control is from the unpublished control catalog of an agency we would be foolish to name here.  Oh, darn, you talked me into it, the agency is the Director of National Intelligence - Extralegal Ventures to Rectify Information Technology Hacks, Incursions and Numbskulls Gabbing (DNI-EVRYTHING):

PS-1337 PERSONNEL SANITIZATION AND DISPOSAL

Control:
The organization sanitizes information system personnel prior to disposal or release for burial.

Supplemental Guidance:
Sanitization is the process used to remove information from information system personnel such that there is reasonable assurance, in proportion to the confidentiality of the information, that the information cannot be retrieved, recovered or extraordinarily renditioned. Sanitization techniques, including clearing, purging, and destroying personnel information, prevent the disclosure of organizational information to unauthorized individuals when personnel are disposed. The organization uses its discretion on sanitization techniques and procedures for personnel containing information deemed to be in the public domain or publicly releasable, or deemed to have no adverse impact on the organization or individuals if released for reuse or disposed. The Black Operations For the Homeland (BOFH) provides personnel sanitization guidance and maintains a listing of approved sanitization procedures in their publication “Leave No Incriminating Evidence (or Where Jimmy Hoffa Went) Directive and BBQ Cookbook”.

Control Enhancements:
(1) The organization tracks, documents, and verifies personnel sanitization and disposal actions.
(2) The organization periodically tests sanitization equipment and procedures to verify correct performance.
(3) The organization employs personnel sanitizers (’cleaners’) who bear an uncanny resemblance to either Harvey Keitel or Jean Reno to perform ad hoc personnel sanitization procedures.
(4) Lbh fubhyq arire gehfg EBG13 rapelcgvba be chg lbhe snvgu va pbafcvenpl gurbevrf. (ROT13 Super-Encrypted)

LOW: Not Selected  MOD: PS-1337(1)(2)  HIGH: PS-1337(1)(2)(3)  MAJESTIC12: PS-1337(1)(2)(3)(4)

Posted in IKANHAZFIZMA, Odds-n-Sods | 1 Comment »

No, FISMA Doesn’t Require That, Silly Product Pushers

July 31st, 2008 by rybolov

Post #9678291 on why people don’t understand what FISMA really isSecure64 DNSSEC Press Releases.

“FISMA Act encourages U.S. government agencies to configure their DNS servers to the DNSSEC security specifications set by the National Institute of Standards and Technology, and it has been reported that the federal governments Office of Management and Budget (OMB) plans to begin enforcing DNSSEC requirements through an auditing process, setting the standard for DNS best practices.”

Yep, if you stamp FISMA on it, people will buy it, maybe in your PR department’s wettest and wildest dreams.  Guys, it’s been 6 years, that kind of marketing doesn’t work nowadays, mostly because we spent ourselves into oblivion buying junkware similar to yours and now we’re all jaded.

Now don’t get me wrong, DNSSEC is a good thing, especially this month.  But there is something I need to address:  FISMA requires good security management with a dozen or so key indicators, not a solution down to the technical level.  Allusions to OMB are just FUD, FUD, and more FUD because unless it’s in a memo to agency heads, it’s all posturing–something everybody in this town knows how to do very well.  OMB would rather stay out of mandating DNSSEC and maybe give a “due date” once NIST has a final standard.

My one word of wisdom for today:  anybody who tries to sell a product and uses FISMA as the “compelling event” has no clue what they’re talking about.

Posted in FISMA, What Doesn't Work | 7 Comments »

« Previous Entries


Visitor Geolocationing Widget: