The Payment Card Industry Security Standards Council (PCI SSC), under pressure from merchants to improve the training of its certified Qualified Security Assessors (QSA), has detailed plans to beef up its PCI QSA certification review process, adding much needed staff and funding to improve oversight of the individuals who conduct PCI Data Security Standard (DSS) compliance assessments.
"I've seen certifications conducted by others that are downright scary; it's alarming what people sign off on."
Jerry Hughes, Certified QSA, Lighthouse Computer Services Inc.
"We're continuing to add people as the program grows," said Bob Russo, general manager of the PCI SSC. "We're making sure that QSA companies have an internal process that they are going through to ensure timely and accurate assessments."
The council, an independent organization founded by five payment card brands, maintains the PCI standards and governs training and approval of QSAs and Approved Scanning Vendors (ASV). It initially had three people reviewing hundreds of PCI DSS assessments for inconsistencies that could signal a troubled or disorganized assessment firm. The organization is hiring a quality assurance analyst and currently has a staff of five reviewing QSA assessments.
Russo wouldn't say how much is budgeted for the remediation program. He said the budget allocation was a significant amount of the PCI Council's
The PCI Council announced its QSA remediation program in 2008 to temper complaints from merchants of inconsistent assessments and, in some cases, certified assessors didn't seem skilled enough to perform a comprehensive PCI DSS assessment. Since then, more than a dozen certified assessment firms have been placed in remediation. Once in remediation, a firm is given 90 days to correct problems and improve its processes. Russo said most comply and are taken out of remediation, but "about a handful," he noted, chose to drop their PCI DSS assessment services completely.
PCI DSS assessments:
MasterCard reverses PCI compliance requirement:
Level 2 merchants do not need to obtain a QSA onsite assessment.
PCI QSA assurance program penalizes assessors: Two firms certified to conduct PCI assessments have been placed into the PCI Council's remediation program for violating the QSA Validation Requirements.
Third QSA firm placed in remediation by PCI SSC: The PCI Security Standards Council quality assurance program placed three QSA firms into remediation. They could face revocation of their certification to conduct PCI assessments.
"I think we're doing what we set out to do and that is to improve the assessment process and make the payment process more secure," Russo said. "I've heard merchants saying that there's a major difference between two years ago and now."
The assessment review process also relies on merchants' feedback to prompt a review of an assessment firm. Merchants are required to fill out feedback forms rating an assessor's technical skills and understanding of PCI DSS. The feedback also addresses QSA ethics; namely whether the assessor implied that a particular commercial product or service was necessary for compliance.
The quality of on-site assessments may become even more critical if Level 2 merchants -- those that conduct between 1 million and 6 million annual transactions -- are forced to undergo on-site assessments. Level 2 merchants make up the largest portion of the payment industry.
Beginning in June 2011, MasterCard will require that merchants conducting a self-assessment questionnaire have staff attend PCI DSS merchant training programs and pass a PCI SSC accreditation program. The card brand recently reversed a controversial decision to require Level 2 merchants to conduct on-site assessments.
In addition, Russo said the PCI Council is reviewing how it trains assessors. Currently, assessors are only required to take a weekend course and in turn typically pass an open-book test. Russo said the program is being modified to include a closed-book exam including essay questions. Annual QSA education for recertification has been formatted to be conducted online.
"There is a background check and they must have a certain level of security industry certifications," Russo said. "They are required to go through training and get requalified every year."
But holding all the available security certifications doesn't necessary provide a security professional with the auditing skills needed to conduct a thorough review of a company's systems, said Jerry Hughes, a certified QSA at Lincoln, R.I.-based Lighthouse Computer Services Inc.
Despite the essay questions, the test is made up of multiple choice questions and focuses on technology areas within the standard. Hughes said the skills he learned conducting audit work for the banking industry, the FDIC and FFIEC and for the federal government helped provide a solid foundation to make a better, more factual determination of a company's PCI DSS certification.
"There's a lot of folks that can break into a network and run scans, but we need to do field work and capture evidence to formulate an opinion," Hughes said. "I've seen certifications conducted by others that are downright scary; it's alarming what people sign off on."
NDB Advisory LLC, an Atlanta-based accounting firm, was asked so frequently by its clients to conduct PCI DSS certifications that it has team of accountants get certified to conduct QSA assessments. An NDB accountant and certified QSA who wished to remain anonymous said he believes the current training provided by the PCI Council enables people to conduct a thorough assessment, but what may be missing is guidance on whether assessments should include a level of quantitative and qualitative analysis.
Some security professionals look at it as a way to conduct a quick assessment and write a report, while others are trying to back up their findings with data, the QSA said.
"I think a lot of QSAs out there can be good consultants; they're savvy technology wise, but I wonder whether they're doing enough to validate their opinions," the QSA said. "When we lack any reasonable amount of guidance, I know we fall back on a traditional accounting audit processes and do more rather than less."