Before selecting a penetration tester, or pen tester, determine exactly which systems you want to test. An expert that tests Unix-based systems is not necessarily also going to be an expert with Windows systems, for example. Once you've decided what to test, I would seek references from colleagues at other companies who have contracted similar work. I prefer this approach rather than relying on pen testing certificates, since there isn't really an industry standard in this field yet.
I wouldn't always just concentrate on the big name consultancies either. These consultants tend to be generalists, and penetration testing is the job of a specialist. Whoever you use, make sure that you are not left with a trainee once the contract has been signed.
It's also wise to inquire about a potential pen tester's favored methodology. The best way to perform penetration testing is to carry out a methodical and repeatable series of tests, working through many different types of vulnerabilities to avoid an inefficient scattergun approach. Be wary, though, of a checklist approach or an over-reliance on automated tools. This style results in more of a vulnerability scan than a full penetration test. Penetration testing is not an exact science so check that the tester has the flexibility to follow up on any areas of concern and pursue the path of least resistance. This way, the test can focus on attack vectors specific to your environment.
Once you have decided who will do the testing, make sure that they have time to complete a thorough evaluation. A tight time constraint may force a tester to skip certain avenues of concern. It is important that they keep you informed of any findings and that the final report details the tests completed, key discoveries and recommendations. Remember that the report is what you are paying for, and you will want time to discuss it with the tester. If you don't take the time up front to properly select your tester, not only will you waste a significant amount of money, but the report you receive may also give an organization a misleading, false sense of security.
This was first published in July 2007