Security: Measuring Up

ROI: Positive Returns

Requires Free Membership to View

Return on investment (ROI), from a business perspective, is elementary: You spend money with the expectation that you'll make back the same amount, plus some revenue gain. An e-commerce server that costs $20,000 and processes $100,000 in transactions will have a $80,000 ROI, or 400 percent.

Security is a cost center like human resources, the mailroom and facilities; it doesn't generate revenue. Even here, though, evaluating ROI is straightforward. You compare actual costs--both known spending and known incident costs--with projected costs, based on assumptions and expectations about what will change with the new product or service. The ROI is savings generated by efficiency. An automated patch management solution, for example, will reduce the time it takes to deploy patches compared to a manual process, resulting in a cost savings.

The same logic is applicable to most any security process: services vs. in-house operations, or help desk vs. manual password resets, IPS and incident reductions. If the savings is greater than the product's TCO, you have positive ROI.

Generally speaking, ROI is gained by increasing the productivity of your IT staff through automation and reducing recurring, predictable incident costs. Frequent incidents and their associated response costs are a factor in some security spending, such as antivirus and antispam, in which ROI is an evaluation of operating costs under different conditions, such as running a network with and without AV.

--Pete Lindstrom

Measuring Security Spending
As part of the overall security management program, it's important to measure total spending and spending effectiveness. Measuring enterprise-wide security spending is more difficult than it appears. Security spending is often divided among various business units and departments, as well as being lumped in with network and infrastructure spending. Finding all of the security dollars and separating them from other budget items is a daunting task.

Sort through the IT chaos using activity-based management by classifying different forms of security activity and correlating all spending related to them. An example of this is the "Four Disciplines of Security Management" (www.spiresecurity.com/fourdisciplines.asp): trust management (policy enforcement, security architecture, certificate management), identity management (user account management, password resets), vulnerability management (configuration hardening, vulnerability remediation) and threat management (monitoring, threat analysis, incident response, forensics). The classification system doesn't matter, as long as you're consistent in your application.

What spending information do you collect for your activity-based analysis? Everything. Spending is broken down into salaries (the percentage of people's time spent on security), hardware and software, and support and maintenance costs.

Staff (people) cost is a matter of identifying wage data for the portion of time anyone spends on security activities. The goal is to identify the percent of time staff members spend on security and calculate it as a percentage of their salaries. This is similar to calculating productivity. After doing this for every individual, the aggregate amount is the total enterprise security spending for personnel.

Allocate these amounts across each of the security activities in the model. The easiest way to do this is to allocate time by percentage for each activity. The percentages can be used to calculate spending for each activity by employee, department or the entire enterprise.

Finally, collect spending for products related to the security activities. The TCO includes product costs, and maintenance and support fees. Dividing TCO by the product's life expectancy is useful to annualize the costs (this is similar to how accountants depreciate and amortize costs over a number of years).

The result is a clear picture of how much an enterprise is spending on security, which can then be broken down by department for comparative purposes. Over time, activity-based analysis will be able to show trends in security spending. And, it's a good method for evaluating security purchases and calculating return on investment (ROI) (see "Positive Returns").

Measuring and Reducing Risk
Traditionally, security managers estimate risk (the probability of something bad happening) as a percentage incorporated into the ALE formula. If the expected loss from a compromise is $100,000, and you estimate that it could happen once every 10 years, the risk is 1 in 10 and the ALE is $10,000.

But which do you pay more attention to: the 10 percent chance of losing $10,000, or the 1 in 100 chance that you'll lose $100,000? Again, risk is often like art--judged by the eye of the beholder.

Defining risk for an entire enterprise is next to impossible. A more manageable approach is breaking it down into small units and types. There are three common forms of risk:

  • Manifest risk is the ratio of malicious events to total events in a computing environment. There are at least four event types: flows, sessions, program commands and transactions. You calculate manifest risk by counting the number of flows and either estimating or using historical data to count the number of malicious events.
  • Inherent risk is the likelihood that system configurations will contribute to a compromise. This is calculated by counting a system's configuration attributes like open ports and services running, and comparing the number of associated vulnerabilities.
  • Contributory risk is a measure of process errors or mistakes made during the normal course of operations and maintenance that contribute to a compromise. Errors associated with user account management procedures and vulnerability management procedures are fertile ground for measuring contributory risk.
Even with these three risk types, you still need to determine the security event/incident frequency. First, you can use historical data to predict the future. If you can identify, for example, the volume of malicious traffic using IDS logs and have already counted the total number of flows, you can project future risk.

Obviously, it will take some time before these numbers are available, and some may never be. But there are other benefits: Even without a number for malicious events, it's possible to measure an environment's exposure.

Exposure is about possibility rather than probability (risk). Exposure measurements can be used as a relative comparison within an environment or across companies. If one can assume that risk is constant for like-sized companies (even if we don't know the number itself), this exposure measure can act as a "risk proxy" to measure the relative difference in risk levels.

Given the elusiveness of quantifiable risk, exposure can provide a means to measure your effectiveness at reducing risk.

None of this is designed to answer all your questions about metrics. But, methodologies outlined for calculating asset value, potential loss, security spending and risk give you a foundation for gathering data and applying it to your own goals and expectations. And, it will bring you closer to assigning values to that security formula and defining what risk means to your enterprise.

This was first published in February 2005

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: