Risk-based audit methodology and background
Risk-based auditing methodology: Risk assessment
The audit universe
The audit plan
Developing a Risk-based audit methodology: A 6-step process
Risk-based auditing case study
Why use a risk-based audit? Methodology and background
Audits are an essential component to an organization's security strategy. They enable staff to meet regulatory requirements, validate that existing controls protect business functions, and determine when new controls are required. Unlike an audit in which the auditor uses a checklist and pen to determine compliance, a risk-based audit requires having an understanding of the organization's business functions and objectives -- to really dig deep within systems and networks.
According to the Information Systems Audit and Control Association, or ISACA, "In a risk-based audit approach, [information systems] auditors are not just relying on risk; they also are relying on internal and operational controls as well as knowledge of the company or the business." Thus, a risk-based audit provides a more thorough assessment of business risk, and enables managers to make informed decisions based on their risk appetites. Aligning enterprise IT decisions and practices with the level of acceptable risk in an organization is the driver for beginning a risk-based audit, and it is the risk assessment process that helps determine that risk threshold.
Risk-based auditing methodology: Risk assessment
Risk appetite or acceptable risk is the amount of risk exposure that a business is willing to accept. That is, the organization must set a threshold for identifying when and where to implement controls to mitigate risk. This process is essential to determining what controls are "nice to have" vs. those necessary to protect business functions. Several risk management methodologies, such as those by the International Organization for Standardization (ISO) and the National Institute of Standards and Technology (NIST), provide managers with good estimates for assessing acceptable risk. In some manner, those methodologies commonly include identifying assets, threats, vulnerabilities and controls.
Organizations cannot protect what they are not aware of, so they must identify all assets, with a sharp focus on those most critical. Once a full list of assets is obtained, the organization must categorize them using some taxonomy, or categorization criteria. In a basic sense, assets can be categorized and assessed by:
Determining criticality and confidentiality levels
Once assets are categorized, the next step is to assign criticality and confidentiality levels. Criticality and confidentiality levels serve to dictate the specific controls for groups of assets, based on confidentiality, integrity and availability requirements. The below criteria offers an easy approach to assigning criticality and confidentiality levels to assets:
Criticality levels based on integrity and availability requirements:
Classification levels based on confidentiality requirements:
Threat and vulnerability identification
After identifying, classifying and assigning criticality levels to assets, the next step is to identify their threats and vulnerabilities. Threats occur because of vulnerabilities . Vulnerabilities are weaknesses in people, processes and technology, and are exploited by threats. For instance, malicious software (or malware) is a threat that can exploit the vulnerability of having non-patched systems. Another potential threat is a black out, which can exploit the lack of having building generators. Threats are categorized by business, technical, physical and administrative. It is important for organizations to not only be aware of their threats and vulnerabilities, but also understand them to determine the level of risk they pose and required countermeasures.
Calculating risk is essential to determining the best use of resources. A simple formula for calculating risk is, risk = asset value x threat x vulnerability. Remember from the description above that the asset value is more than its initial cost. The value of an asset increases as resources are applied, and as competitors value it. Resources are applied towards assets in development, testing, operations and maintenance. In addition, the information within an asset is of value. For example, let's look at a Web server that accepts customer information. If the database that contains customer information is compromised, its loss may cause legal penalties and reputation damage. In addition, loss of trade secrets can cause a decrease in market share and in competitive advantage. Therefore, asset value must take into account all factors that affect its value.
Calculating a threat value requires looking at its estimated projected loss (PL) and annual rate of occurrence (ARO). PL is the percentage exposed by the asset if the threat occurs. That is, if malware compromises the database that holds customer information and 60% data loss is anticipated, then PL is 60%. We then look at the threat's ARO by determining how often a threat is projected to occur within a single year. In this case, the projection may be an annual rate of occurrence of once every two years, or .50. This would make our threat calculation .3 (PL x ARO).
The last calculation before determining risk is deficiency level (DL). DL is the amount of protection deficient given controls already in place. That is, if existing antimalware technology in place to protect the company's database is 80% effective, then there is a 20% deficiency level. Using this figure, along with asset value and threat calculations, we can calculate risk. In this example, let's use $500,000 as the value of the database. In such as case, risk = $500,000 x .3 x .20 = $30,000. Under this scenario, we can justify an investment of up to $30,000 to protect the critical database.
The risk assessment process enables audit teams to prioritize engagements based on risk, while remaining focused on critical assets that significantly affect business operations. This risk assessment process is coupled with ranking risks in the audit universe. Remember, these exercises require some estimations, since each attack or threat vector is different.
The audit universe
The audit universe includes all potential audit entities and processes that may be assessed by the audit team. Essentially, the audit universe includes the people, processes and technology that drive the organization's business objectives. Examples of potential areas within the audit universe include infrastructure, applications, processes, architectures, regulatory compliance, frameworks, policies and boundary protection. Along with determining risk in monetary terms, another way is by ranking risk using a risk table.
Risk ranking is the process of using judgment to score audit entities. The risk ranking table is calculated using a concern rating scale and the value assigned to each risk area. For instance, the table below regards Web applications as a high-risk area. The table lists Web application complexity as a high concern, which has a concern rating of three. The value assigned to complexity is 1.75; therefore, 1.75 x 3.0 = 5.25, the risk value for Web applications complexity.
The table above provides a high-level example of ranking risk. Given the ranking criteria, the top 25% of audit entities is considered high risk and must be audited in that year. According to this table, all Web applications and Web servers must be audited this year. A more comprehensive risk ranking table could rank specific Web applications, or even each Payment Card Industry Data Security Standard (PCI DSS) requirement as its own auditable entity. The level of detail in listing entities is dependent upon the size of the audit universe and the resources available to perform an in-depth risk ranking assessment. The annual risk assessment and ranking process determines the audit plan.
The audit plan
The audit plan outlines the annual audit schedule, scope, objectives and resources required to start audit engagements. The audit plan is created using risk assessment results and risk ranking. The risk assessment truly drives this process as it helps the audit team identify deficiency levels, unaddressed weaknesses, and areas of concern. In addition to the risk ranking criteria shown in the table above, other potential criteria are recent technology refresh, recent mergers, recent acquisitions, new regulations, etc. The audit plan defines roles and responsibilities, the audit team's methodology, logistics and performance measures. Essentially, the audit plan serves as the road map -- with management's approval -- for the audit team to start conducting audits.
Developing a risk-based audit methodology: A six-step process
After the annual process of identifying and ranking risk, and developing the audit plan, the audit team can "roll their sleeves back" and begin auditing. The auditing process involves a phased, six-step risk-based process that includes preparation, assessment, mitigation, reporting and follow-up. Each phase is explained below.
The preparation phase involves individual engagement planning before each audit in the audit plan. During this phase, the audit team reviews previous working papers, risk assessment findings, and defines the individual engagement's scope and objectives. The team will listen to the concerns of management and their operators and then compare that information with their assessment results. The audit team's focus is to understand the business objectives and functions of the company to make the most informed assessment.
The assessment phase involves analyzing systems and process, identifying vulnerabilities and documenting concerns. During this phase, the auditor may use a checklist, but will rely heavily upon experience and judgment to interpret results and identify less-noticeable anomalies. During this phase, it's important to collaborate with operators to validate the significance of risk. Certain "textbook" findings may seem significant at first, but may be in place for good reason (such as unique operational requirements for certain applications). Therefore, it is important to validate certain results before causing too much stir by listing it as significant on the final report.
Depending on the system, network, or process being audited, some baseline controls exist that they can be assessed against. For instance, the SANS Institute outlines a prioritized list of baseline security controls and measures. These controls were developed by SANS, in collaboration with federal agencies, civilian penetration testers and forensic experts. Some of these controls include inventorying authorized and unauthorized devices, boundary defense, application security, malware defense, data loss prevention, account control, wireless control and data recovery capabilities. In addition to auditing against baseline controls, it is important to ensure controls are in place that enable business functions.
The mitigation phase involves developing the proper controls to mitigate risk. This process involves socializing requirements with the company and developing mitigation plans. While some controls may take weeks or months to implement, others can be rectified on the spot. This phase involves documenting potential controls and the actions taken by operators to mitigate risks on the spot.
In the reporting phase, the audit team provides a full report to management outlining its findings. The group will share mitigation plans for ongoing controls, and outline the most significant findings. This phase involves developing an executive summary with key information for managers to make security decisions. The executive summary is a high-level overview that explains "in a nutshell" the security posture of the organization and the next steps required to strengthen their controls.
In the follow-up phase, the audit team corresponds and works with the company to ensure controls are implemented. In a resource-constrained environment, the company may lack the skill sets to implement certain control mechanisms. Therefore, the auditors may provide insight into implementing various controls, and will continue to follow up to ensure their implementation. It is important for the audit team to have a process for tracking overdue or upcoming mitigation implementations. The team should create a tracking system that contains an easy-to-follow dashboard that provides the status of implementations and their due dates.
Hypothetical case study: Implementing a risk-based audit methodology
Now that we understand what auditing is all about, let's explore a use-case scenario that brings it all together. In this example, we walk through, at a high level, the audit process for an office supply franchise that specializes in office materials, office furniture, logo and print media services. This company accepts credit cards within its 12 stores across the East Coast, and through e-commerce using their online storefront. We will call this company Office Company.
Office Company identified all assets and categorized them by type, value and complexity. It used the risk ranking table below to rank its risk using the listed criteria.
After ranking a few audit entities within its audit universe, Office Company found that its primary concerns are corporate firewalls and PCI DSS compliance. Specifically, it is concerned with the "Build and maintain a secure environment" PCI DSS requirement because it recently lost its primary firewall administrator. The company classified its firewalls as confidential (requiring strict controls to prevent unauthorized access), with criticality level 1 (requiring a high level of protection for vital sustainment of business operations). Within its audit plan, the audit team decides to focus a lot of effort toward its corporate firewalls and the above PCI DSS requirement.
During the preparation phase, the auditors find that the former firewall administrator rarely documented firewall changes, had not updated the network topology in several months, and left junior staff untrained. In order to protect e-commerce data and prevent regulatory penalties, the audit team agrees to perform a full scrub of each firewall's rule set. It intends to assess ingress and egress filtering, including the documentation of each rule, and all blocked and allowed ports, protocols and services.
During the assessment phase, the audit team finds that the firewalls used within the enterprise are at their end-of-life and close to end-of-support by the vendor. This was just one of many findings, but the most significant that required immediate attention. The audit team worked with management to determine the value of the firewall assets and cost required to update them. In the mitigation phase, the group determines that the risk of not replacing the corporate firewalls is too costly.
Office Company valued its enterprise firewalls at $1.5M. However, the "end-of-support" circumstances increase the threat and vulnerability calculations to .4 and .7, respectively, making the risk level $420,000. This was due to an increase in the threat (projected loss and annual rate of occurrence) once vendor support is gone, and an increase in deficiency once vendor support is lost. The Office Company sets a $400,000 budget to cover maintenance contracts, training, testing and deployment of the new firewall product. The group then decides to invest an additional $20,000 to hire an external company to test and validate the effectiveness of its controls.
In the reporting phase, the audit team recaps the corporate firewall and PCI DSS concerns with the management team. They cover the next steps required to begin the firewall upgrades and follow-up milestone dates. To provide positive points, they report some of the shortfalls in the existing firewall architecture that the administrators were able to correct on the spot. One area in particular was the "default deny" rule on the firewall. As recommended by Dr. Anton Chuvakin in his new book, PCI Compliance, the audit team had the firewall administrators implement a "stealth rule" on the firewalls to drop all inbound and outbound traffic targeting the firewall device itself. In addition, it covered other areas found during the audit, including use of an authentication server on infrastructure devices and contingency plan changes.
This guide highlights the core areas of a risk-based audit. From the risk assessment and risk ranking process, to assessing and reporting findings, it is important to take a risk-based approach focused on enabling business objectives. This model is iterative and ongoing to keep up with the evolution of technology and ever-present vulnerabilities. It is important for the audit team to be mindful of the amount of estimation used in the risk ranking process. Auditors must be realistic in their analysis of threats and risks, and use measurable numbers when possible. Using this approach on an annual basis enables organizations to achieve enterprise security.
About the author: Marcos Christodonte II, MBA, CISSP is an information security professional working for a consulting firm. He is the author of Cyber Within: A Security Awareness Story (and guide) for Employees. Marcos can be reached at his website: www.christodonte.com
This was first published in April 2010