Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

What are the pros and cons of zero-knowledge penetration tests?

A penetration tester with no previous knowledge of the site being tested may be able to give some insight unavailable to other forms of penetration testing, but there are pros and cons. Expert Michael Cobb weighs in.

What are the pros and cons of penetration tests where the tester has zero knowledge of the website being tested? And should testers ideally have zero knowledge (to better simulate an attacker's mindset)?
Zero knowledge in the context of a penetration test on a website means that the penetration tester is told very little about the target, maybe as little as its URL, thereby simulating real-world attackers.

While it can be helpful, perhaps in the context of budgeting and office politics, to present your boss with a report that proves even someone who has no inside knowledge of the new website can hack into it, I have several reservations about the zero-knowledge approach. We know that a certain percentage of attacks are going to come from inside the network perimeter, or from the outside with insider help. If you want to know how secure your site is across all real-world scenarios, zero knowledge is not necessarily the best starting point.

A zero-knowledge approach also has the potential drawback of being slower to return results. If you describe some of the basics of your system to the tester beforehand, it can save time, and time is often tight when a new product is being rolled out. One important variable here is the status of the target: is it in production or in development? When testing a production system, you may want testers to let you know about a gaping hole as soon as it is discovered, rather than waiting until the final report. Provided the contract with the tester is appropriately worded, you may be able to patch the hole and get the patch tested. Indeed, some would argue that treating a pen-test as an iterative improvement in security is better bang for the buck.

Finally, whether you choose to proceed from a zero-knowledge starting point, remember that you can't truly replicate the real world without breaking the law. You must assume your attackers are prepared to commit illegal acts to achieve their ends, but few organizations are in a position to give their pen-testers a get-out-of-jail-free card. So, you will want your pen-tester to be able to think like a criminal hacker and document for you those methods of penetrating the system that rely on illegal acts.

The bottom line is that the real world and a pen-test are two different things, and your security money may be best spent having seasoned security experts explore the potential vulnerabilities of your product while armed with plenty of knowledge about it, rather than setting up unrealistic testing scenarios.

More information:

This was last published in May 2008

Dig Deeper on Penetration testing, ethical hacking and vulnerability assessments