Essentially, there are three different types of security diagnostics: penetration tests, audits and assessments (variously described as vulnerability assessments, threat assessments or risk assessments). No single test is automatically always the best one to perform. In measuring systems security, you have to be careful to perform the right test at the right time. It's also important that the selected test be based upon the actual needs of the organization, not the skills (or lack thereof) of the testers (whether they be in-house employees or outside consultants).
A penetration test has the most sizzle, since everybody's heard of it and "knows" that a penetration test is what the pros use to make sure a system is secure. Penetration tests are sexy nowadays, but in most circumstances they are actually the least useful of the various systems diagnostics.
First things first: A correctly performed penetration test is a covert test, in which a consultant or trained insider plays the role of a hostile attacker who tries to compromise systems security. Since the ultimate goal is penetration, the test is carried out without warning, in something close to complete secrecy (of course, upper management has approved the test and understands the need for secrecy). Ideally, there should be no support from the organization being tested (read: attacked)... or, at most, guidance should be restricted to what the penetration team should avoid. Obviously, if the organization is outsourcing the test, the client should let the consultant know what the specific goals are. The test can be designed to simulate an inside or outside attack. It can also be technical or non-technical (e.g., the tester may try to social engineer his or her way into the network). Only a few people inside the target organization should know about the test. A critical aspect of the test should be to see if the organization can detect the penetration attempts. For that reason, the people who would authorize a formal response should also be involved.
Now, why isn't a penetration test as useful as it's made out to be? Because its only goal is to compromise security. To do this, the team identifies potential holes, with emphasis on the ones they believe are the most fruitful and least likely to be detected (from the attacker's perspective). At that point, the client sees what potential damage could result from the exploitation of these vulnerabilities. However, in running the test, a tester does not find all the system's vulnerabilities, and does not even confirm the existence of all the vulnerabilities that the test may have detected. All a penetration test proves is that a system can be compromised. It does not document every weak point, just those that may have been exploited during the test. Thus, while certain other issues may be extrapolated from the penetration, no penetration tester can claim to have comprehensively identified all of the client's security problems -- or even a majority of them.
So, what is a penetration test good for? Well, for a variety of internal reasons some organizations need a convincing argument that inadequate security may cause significant loss. A well-performed penetration test can certainly demonstrate this. To make the test useful from a business perspective, the potential loss of corporate value must be demonstrated dramatically and vividly, above and beyond the bare fact that the company's computers can be compromised.
Sometimes, a covert penetration test should be performed just to see if security policies are being followed. While a publicized test could study whether or not people follow policies, it's only human nature to behave differently when you don't know you're being watched. For instance, let's say that XYZ Corp's security policy prohibits end users from revealing their passwords over the telephone unless they themselves have initiated a call to the IT help desk. Obviously, if an outside consultant walked up to an end user and asked, "Do you ever reveal your password to anyone you don't know," the answer would usually be no. But it's a different story if the tester calls up a user, pretends he's from the IT department, and asks the user to reveal his or her password so the tester can "confirm" it. Such social engineering penetration techniques are a much more reliable method of determining actual compliance with security policy.
Read the rest of this article.
About the author
With more than 17 years of experience in the intelligence and security fields, Ira Winkler is the chief security strategist for HP Consulting, North America. In this role, Ira helps determine client needs and provide advice on security strategies and implementation. He serves on various industry advisory committees and consortia to further demonstrate leadership in the Internet security field.
This was first published in December 2003