Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

PCI DSS: Why vulnerability assessment and penetration testing are so hard

Enterprises fail to meet requirement 11 more than any other PCI DSS requirement. Expert Mike Chapple explains why and how firms can do better.

The 2014 Verizon PCI Compliance Report assessed the state of PCI DSS compliance around the world. Surprisingly,...

it found that Requirement 11, which calls for the regular testing of security systems and processes, was the least complied with, despite the fact that many security professionals consider it one of the more straightforward provisions in the report.

This tip explores the two specific areas that Verizon highlighted as being stumbling blocks to compliance and how they can be built into a PCI DSS compliance program.

PCI DSS requires penetration testing of an environment following an industry-accepted approach, such as NIST SP 800-115.

Penetration testing
It's well known by security professionals that PCI DSS requires penetration testing of an environment following an industry-accepted approach, such as NIST SP 800-115. These tests must be performed at least on an annual basis, and also must be repeated after any major changes to the cardholder data environment.

The first issue Verizon discovered is that 60% of organizations failed to provide evidence that they had conducted penetration testing within the past year. Maintaining documentation of the penetration test is almost as important as actually conducting the test when it comes to PCI compliance. Auditors are not willing to take your word for it; they want to see evidence.

The second area where a company may encounter problems is in handling the results of penetration testing. PCI requires firms to act upon the findings from penetration tests. Specifically, if the testing discovers any exploitable vulnerabilities, the organization must correct the problem and then repeat the test. The penetration testing cycle is not complete until vulnerabilities are amended and a penetration test provides a clean bill of health. According to the Verizon PCI Compliance Report, only 44% of firms satisfied this requirement.

It's possible to steer clear of penetration testing landmines by ensuring that the penetration testing program complies with the letter of the PCI law. It's not necessary to hire a QSA or ASV to perform the test. If you instead use internal resources, ensure and maintain evidence that they are both qualified and organizationally independent from the staff maintaining systems. This is an excellent role for an internal audit team if there is a qualified staff member.

Vulnerability scanning
Vulnerability scanning is another area where many organizations stumbled on PCI compliance. For example, less than half of organizations in the study performed internal vulnerability scans or external scans conducted by an approved scanning vendor -- both of which are necessary requirements for PCI. These scans must occur inside the cardholder environment and may be administered by either an external assessor or by internal staff who are not responsible for maintaining the systems being scanned.

When preserving the scanning documentation, keep records that clearly demonstrate a clean scan result every three months. It is not sufficient to retain only the most recent scan record, as this only shows point-in-time compliance instead of the required year-round scanning program. Also, the scan results must be clean and must be administered each quarter. Any high-risk vulnerabilities that are detected must be remediated as soon as possible, and follow-up scans should be conducted until all issues have been successfully resolved.

The Verizon report doesn't detail the specific reasons that companies fail this test, but it's possible that companies get tripped up by inadequate recordkeeping, inappropriately using staff who maintain security controls to run the scans or failing to run scans repeatedly until they are clean. This would be a good time for a company to check on its scanning program to make sure it has appropriate paperwork for both internal and external scans.

Despite these struggles, the world of PCI compliance has come a long way during the past year. The optimistic highlight of Verizon's research revealed that the average compliance level for companies subject to PCI DSS rose from 52.9% of controls in 2011 to 85.2% of controls in 2013.  That's a major step forward and a good sign that PCI DSS compliance is becoming a part of routine operations by fortifying enterprise security postures and reducing the likelihood of a costly card data breach.

About the author:
Mike Chapple, Ph.D., CISA, CISSP, is senior director for IT service delivery at the University of Notre Dame. He previously served as an information security researcher with the National Security Agency and the U.S. Air Force. Chapple is a frequent contributor to SearchSecurity, and serves as its resident expert on enterprise compliance, frameworks and standards for its Ask the Experts panel. He previously served as site expert on network security, and is a technical editor for Information Security magazine and the author of several information security titles, including the CISSP Prep Guide and the Security+ Training Kit.

This was last published in June 2014

Dig Deeper on PCI Data Security Standard

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Why do you think enterprises fail to meet PCI DSS requirement 11 (regular testing of security systems and processes) more than any other requirement?
I suspect there are so few cybersecurity professionals really qualified to do this on a regular basis, that its hard to see this done on a regular basis.
We have 2 people sharing the responsibility to comply with the pentest and vulnerability scans. Why is that we have to do our part as SMB and then see national retailers, banks, telcos and government agencies get compromised. Shouldn't they have more stringent standards than us as they are higher profile/potential targets?