With a penetration test, it's absolutely critical to ensure that you have skilled people performing it. The quality of the testers always drives the quality of the results. The more skilled the tester, the more likely it is that you will get a complete picture.
But, in addition, penetration tests have great potential for creating damage. While other tests are generally performed in conjunction with the organization's staff, any damage caused by covert testing may go unnoticed for some time.
In deliberately compromising systems, a penetration tester must be careful not to break anything. After all, the work consists of violating standard user policies, monkeying with operating systems, changing permission structures, etc. If someone is careless, unskilled or unethical, he or she could cause a tremendous amount of damage. Also, should one of the client's employees detect the penetration test, the tester has to ensure that there is no overreaction by people who do not know that the attack is part of a legitimate, management-approved project. In particular, neither the tester nor the client wants law enforcement called unnecessarily, since such an event can be extremely embarrassing.
This leads to a question that just about everyone considering a penetration test asks: "Should I hire hackers to perform penetration tests?" In my opinion, the answer is always no. I can understand the reasoning behind the question: Who better to test your system than the kind of people who might break in? But when you think about it, this line of reasoning is inherently flawed: Does a gun expert necessarily know how to make the best bulletproof vest? No. Also, just because hackers may know a few tricks, they rarely know all of the tricks. More importantly, they may lack sophistication in important business matters, such as how to interpret the results of the test in terms of the organization's goals or business strategies. Such individuals are unlikely to be able to provide useful reports to the company, or translate technical details into a language that business people can relate to.
You must look at prospective testers' rÉsumÉs and references critically. If a person claims you should hire him merely because he is a hacker (or "reformed" hacker, or "ethical" hacker), that implies a lack of skills or experience in real-world business settings (which, after all, is what you're operating in). For the same reason, you should also ask security professionals to provide evidence of their skills and references.
If a self-proclaimed "ethical hacker" can prove he has legitimate (and appropriate) skills and experience, you may still want to do a background check to see if you can really trust him with your critical information. The term "ethical hacker" is poorly defined, so some highly qualified and ethical people may call themselves hackers. But while ethical hacking is all the rage, be aware that these highly qualified people are few and far between.
About the author
With more than 17 years of experience in the intelligence and security fields, Ira Winkler is the chief security strategist for HP Consulting, North America. In this role, Ira helps determine client needs and provide advice on security strategies and implementation. He serves on various industry advisory committees and consortia to further demonstrate leadership in the Internet security field.
This was first published in December 2003