# Security: Measuring Up

## Metrics are the key to measuring security. These frameworks will help you gather data and calculate the answers you need.

POLICY & PROCESS Metrics are the key to measuring security. These frameworks will help you gather data and calculate...

the answers you need.

Here's a formula most security pros will recognize:

Risk = Threat x Vulnerability x Expected Loss

It's useful for expressing the necessity and purpose of security. It's also equally difficult to quantify with meaningful numbers.

How do you numerically express a threat? What is the cost of a vulnerability? How do you calculate expected loss? And, when you multiply these three variables, how do you denote risk in a way that can be translated into an action item?

Security metrics--the measure of security policies, processes and products--is the much-sought-after solution to this conundrum. Security managers look for a magic formula that calculates risk and effectiveness in reducing risk, but the reality is that security metrics aren't that simple.

Measuring security is about using common sense. Managers need to determine what to measure, organize the variables in a way that makes them manageable and meaningful, and build repeatable formulas that show the snapshot status of security and how it changes over time.

We'll define some of the underlying elements required for security metrics and how security managers can use them to quantify different portions of their security programs. We can't provide all the answers, but we'll provide you with a better understanding of the complexity of security metrics and insight into how you can leverage existing data to strengthen performance measurements.

Calculating Asset Value
Enterprises routinely place values on all information assets (hardware, software and data) that are reflected in IT spending. Enterprises expect every hardware and software installation to return an amount that's at least equal to its total cost of ownership (TCO) over its life expectancy.

If we accept that premise, we can assume that the minimum value of all computing assets is the amount of IT spending for a year (salaries, operations and maintenance) plus the depreciation or amortization value of the assets (hardware and software). An ERP system with a TCO of \$20 million must return at least that much over its lifespan.

But such calculations need context. That's why we assign quantifiable values to information assets for objective evaluation and comparison. The following are some ways to classify your information assets' value:

Productivity Value. An asset's worth is at least as much as the costs of implementing, maintaining and using it. For a single PC, the minimum information asset value is the cost of the PC plus software, a percentage of IT overhead costs (e.g., help desk) and the user's time (salary). As you'll see later, productivity value is essential for calculating other security metrics.

Revenue Value. For some assets, worth is measured in the value of transactions. If your e-commerce Web server processes \$1 million in transactions a day, it's worth \$365 million annually.

Revenue value isn't always clear-cut. Supply-chain systems, such as manufacturing equipment and control systems, don't generate revenue, but your revenue generators are useless without their productivity. Their value is often calculated by measuring how much revenue would be lost if those systems were unavailable.

What isn't as easy to calculate is the revenue value of a pure information asset, such as software, music, movies and electronic documents, that can be perfectly replicated. The assets' values are usually much higher than their assigned prices, since they're sold many times over. The value of an "information asset for sale" can be estimated using historical data and/or revenue trend information.

Liquid Financial Assets Value. Those much-vaunted "assets under management" figures associated with financial institutions provide a straightforward way to assess their value. If \$1 billion is under management, that amount, plus the productivity value, provides the total value that's being protected. Add to that the revenue values of transaction assets, and you've got the total value of information assets that require protection.

Intellectual Property Value. This is the most difficult asset to value. It's usually seen as "the reason" a company is in business. There are books with elaborate formulas to calculate intellectual property value; however, it may be easiest to consider intellectual property's contribution to a company's market capitalization. This value can be calculated by multiplying the amount of intellectual property captured on systems with the difference between market capitalization and book value.

Calculating Potential Loss
Calculating potential losses is a little different than measuring asset value. Information assets aren't usually "lost" in the traditional sense--they're often still available for use, and partial (or even greater) value may be lost. Some information, such as Social Security numbers, don't have an inherent value; music files that are sold multiple times will have a greater asset value than face value because of their potential revenue. Value is a matter of context.

Because asset value is linked, but not tied, directly to loss, you must consider the type of compromise when evaluating potential losses. There are five distinct types:

Confidentiality breaches occur when high-value information, such as trade secrets or financial data, is read from storage, sniffed from the network by unauthorized parties, or leaked by internal users.

Integrity breaches occur when data is modified in transit or at rest, such as with transactions that are modified to reflect inappropriate quantities or dollar values.

Availability breaches are when information is deleted or made unavailable.

Productivity breaches occur when resources are disabled, such as when an e-mail server is disabled by spam or an e-commerce service is DOS'd by transaction requests.

Liability breaches occur when systems and data are still available, but are being misused. This could be an employee using corporate resources to host a porn site or a hacker storing stolen files on a compromised server.

Determining information asset value enables enterprises to focus on their real security needs and allocate adequate resources. But asset value isn't the only thing that may be lost. It's important to consider incident costs with the lost asset value when evaluating actual and potential losses. Incident costs are the lost asset value that are specifically associated with the incident, including IT productivity, legal and regulatory costs.

The severity of an incident and its losses are calculated by correlating the type of breach with the value type. For instance, if a worm disables your network, you have an availability and productivity breach that affects the asset values of everything on the network--productivity and revenue. The loss equals productivity and revenue values, plus incident costs associated with IT productivity. Legal costs and regulatory fines are also a factor.

A popular method of calculating potential losses is annual loss expectancy (ALE), an estimate for expected loss calculated by multiplying the probability of a particular type of loss by the total loss potential. For example, if there's a 10 percent chance of losing \$1 million, the ALE is \$100,000.

Obviously, there's a difference between a 1-in-10 chance of large loss and the strong likelihood of a number of incidents throughout the year. One way to address this difference is by treating ALE based on the category: recurring losses or specific security breaches. These distinctions allow you to rate the viability of projections based on historical data. If you know you've suffered an average 100 malware infections a month (recurring), you know your projections have a greater validity. Predicting less-frequent attacks is much more ambiguous, making such calculations more art than science.

ROI: Positive Returns
 Return on investment (ROI), from a business perspective, is elementary: You spend money with the expectation that you'll make back the same amount, plus some revenue gain. An e-commerce server that costs \$20,000 and processes \$100,000 in transactions will have a \$80,000 ROI, or 400 percent. Security is a cost center like human resources, the mailroom and facilities; it doesn't generate revenue. Even here, though, evaluating ROI is straightforward. You compare actual costs--both known spending and known incident costs--with projected costs, based on assumptions and expectations about what will change with the new product or service. The ROI is savings generated by efficiency. An automated patch management solution, for example, will reduce the time it takes to deploy patches compared to a manual process, resulting in a cost savings. The same logic is applicable to most any security process: services vs. in-house operations, or help desk vs. manual password resets, IPS and incident reductions. If the savings is greater than the product's TCO, you have positive ROI. Generally speaking, ROI is gained by increasing the productivity of your IT staff through automation and reducing recurring, predictable incident costs. Frequent incidents and their associated response costs are a factor in some security spending, such as antivirus and antispam, in which ROI is an evaluation of operating costs under different conditions, such as running a network with and without AV. --Pete Lindstrom

Measuring Security Spending
As part of the overall security management program, it's important to measure total spending and spending effectiveness. Measuring enterprise-wide security spending is more difficult than it appears. Security spending is often divided among various business units and departments, as well as being lumped in with network and infrastructure spending. Finding all of the security dollars and separating them from other budget items is a daunting task.

Sort through the IT chaos using activity-based management by classifying different forms of security activity and correlating all spending related to them. An example of this is the "Four Disciplines of Security Management" (www.spiresecurity.com/fourdisciplines.asp): trust management (policy enforcement, security architecture, certificate management), identity management (user account management, password resets), vulnerability management (configuration hardening, vulnerability remediation) and threat management (monitoring, threat analysis, incident response, forensics). The classification system doesn't matter, as long as you're consistent in your application.

What spending information do you collect for your activity-based analysis? Everything. Spending is broken down into salaries (the percentage of people's time spent on security), hardware and software, and support and maintenance costs.

Staff (people) cost is a matter of identifying wage data for the portion of time anyone spends on security activities. The goal is to identify the percent of time staff members spend on security and calculate it as a percentage of their salaries. This is similar to calculating productivity. After doing this for every individual, the aggregate amount is the total enterprise security spending for personnel.

Allocate these amounts across each of the security activities in the model. The easiest way to do this is to allocate time by percentage for each activity. The percentages can be used to calculate spending for each activity by employee, department or the entire enterprise.

Finally, collect spending for products related to the security activities. The TCO includes product costs, and maintenance and support fees. Dividing TCO by the product's life expectancy is useful to annualize the costs (this is similar to how accountants depreciate and amortize costs over a number of years).

The result is a clear picture of how much an enterprise is spending on security, which can then be broken down by department for comparative purposes. Over time, activity-based analysis will be able to show trends in security spending. And, it's a good method for evaluating security purchases and calculating return on investment (ROI) (see "Positive Returns").

Measuring and Reducing Risk
Traditionally, security managers estimate risk (the probability of something bad happening) as a percentage incorporated into the ALE formula. If the expected loss from a compromise is \$100,000, and you estimate that it could happen once every 10 years, the risk is 1 in 10 and the ALE is \$10,000.

But which do you pay more attention to: the 10 percent chance of losing \$10,000, or the 1 in 100 chance that you'll lose \$100,000? Again, risk is often like art--judged by the eye of the beholder.

Defining risk for an entire enterprise is next to impossible. A more manageable approach is breaking it down into small units and types. There are three common forms of risk:

• Manifest risk is the ratio of malicious events to total events in a computing environment. There are at least four event types: flows, sessions, program commands and transactions. You calculate manifest risk by counting the number of flows and either estimating or using historical data to count the number of malicious events.
• Inherent risk is the likelihood that system configurations will contribute to a compromise. This is calculated by counting a system's configuration attributes like open ports and services running, and comparing the number of associated vulnerabilities.
• Contributory risk is a measure of process errors or mistakes made during the normal course of operations and maintenance that contribute to a compromise. Errors associated with user account management procedures and vulnerability management procedures are fertile ground for measuring contributory risk.
Even with these three risk types, you still need to determine the security event/incident frequency. First, you can use historical data to predict the future. If you can identify, for example, the volume of malicious traffic using IDS logs and have already counted the total number of flows, you can project future risk.

Obviously, it will take some time before these numbers are available, and some may never be. But there are other benefits: Even without a number for malicious events, it's possible to measure an environment's exposure.

Exposure is about possibility rather than probability (risk). Exposure measurements can be used as a relative comparison within an environment or across companies. If one can assume that risk is constant for like-sized companies (even if we don't know the number itself), this exposure measure can act as a "risk proxy" to measure the relative difference in risk levels.

Given the elusiveness of quantifiable risk, exposure can provide a means to measure your effectiveness at reducing risk.

None of this is designed to answer all your questions about metrics. But, methodologies outlined for calculating asset value, potential loss, security spending and risk give you a foundation for gathering data and applying it to your own goals and expectations. And, it will bring you closer to assigning values to that security formula and defining what risk means to your enterprise.

This was first published in February 2005

## Content

Find more PRO+ content and other member only offers, here.

Oldest

## SearchCloudSecurity

• ### Cloud data classification services: How they benefit organizations

Data classification services from CSPs are important for organizations strengthening their cloud security posture. Expert Dave ...

• ### How the cloud can help organizations with security log data

Organizations often have to deal with copious amounts of security log data and that can be challenging. Expert Frank Siemons ...

• ### DDoS mitigation services: What to consider before implementation

Before implementing DDoS mitigation services, there are a few things enterprises should consider. Expert Ed Moyle discusses the ...

## SearchNetworking

• ### AT&T seeks collaborators on 5G technology

AT&T's partner approach to deploying 5G technology is seen as significantly different from rival Verizon's work with vendor ...

• ### IT professionals aim to operationalize threat intel in the enterprise

This week, networking bloggers look into operationalizing threat intel, Google Fiber's deployment and home lab spending.

• ### A comprehensive review of network performance monitors

Network performance monitors play a crucial role in today's data centers. In this guide, expert Andrew Froehlich helps you ...

## SearchCIO

• ### WhatsApp messaging app comes of age with user privacy shift. Mazel tov?

WhatsApp's new privacy policy may dismay some users. For businesses, it signals the popular mobile messaging app has come of age....

• ### Mobile security issues and solutions

Mobile technology usage is growing just as the number of threats facing mobile data is growing. CIOs must combat mobile security ...

• ### State of cloud infographic: Cloud computing stats for CIOs, IT execs

Surveys show that many enterprises plan to increase or maintain cloud spending in 2016. View our infographic for the latest cloud...

## SearchConsumerization

• ### Android, Windows tablets from HP take aim at business users

HP released a new line of tablets targeting business users. The HP Pro Slate 8 and Pro Slate 12 run Android and cost \$449 and ...

• ### Microsoft to lay off 18,000, Nokia X moves to Windows Phone

Microsoft will lay off 18,000 people over the next year while the Nokia X line of Android smartphones, which was unveiled earlier...

• ### Microsoft Surface Pro 3 vs. Microsoft Surface Pro 2

Surface Pro 2 and Surface Pro 3 are different enough that Microsoft is keeping both on the market as competing products. Which ...

## SearchEnterpriseDesktop

• ### Windows OS updates get simpler -- but there's a trade-off

Microsoft's new monthly rollups will make Windows OS updates less hectic. IT won't be able to pick and choose which updates and ...

• ### Get to know new versions of Microsoft’s Office productivity suite

The newest versions of Microsoft's Office productivity suite are an interesting departure from the Office of old. There are ...

• ### What to look for in client management tools

Bringing PC and mobile device management under the same umbrella is clutch for IT today. Shops can get that done by picking the ...

## SearchCloudComputing

• ### Rackspace acquisition positions company for more managed services

The Rackspace acquisition by Apollo Global Management will make the company privately held and infuse it with cash to double down...

• ### Read between the lines of providers' cloud security assessments

Certifications are a good starting point to evaluate a cloud provider's security, but users need to dig deeper than a simple ...

• ### OpenStack scalability still tests the patience of IT pros

The constant evolution of OpenStack components hinders the platform's ability to scale. Learn why OpenStack scalability ...

## ComputerWeekly

• ### Department for Work and Pensions to recruit chief data officer

The Department for Work and Pensions is looking to hire a chief data officer with a potential salary of around £130,000

• ### Information technology skills guide

Author and IT consultant, Terry Critchley, looks at potential solutions to the apparent skills crisis facing the IT industry, in ...

• ### Rackspace sold to private equity house Apollo Global for \$4.3bn

After weeks of speculation about its acquisition status, Rackspace confirms definitive agreement with private equity buyer

Close