Security: Measuring Up

Metrics are the key to measuring security. These frameworks will help you gather data and calculate the answers you need.

POLICY & PROCESS Metrics are the key to measuring security. These frameworks will help you gather data and calculate...

Here's a formula most security pros will recognize:

Risk = Threat x Vulnerability x Expected Loss

It's useful for expressing the necessity and purpose of security. It's also equally difficult to quantify with meaningful numbers.

How do you numerically express a threat? What is the cost of a vulnerability? How do you calculate expected loss? And, when you multiply these three variables, how do you denote risk in a way that can be translated into an action item?

Security metrics--the measure of security policies, processes and products--is the much-sought-after solution to this conundrum. Security managers look for a magic formula that calculates risk and effectiveness in reducing risk, but the reality is that security metrics aren't that simple.

Measuring security is about using common sense. Managers need to determine what to measure, organize the variables in a way that makes them manageable and meaningful, and build repeatable formulas that show the snapshot status of security and how it changes over time.

We'll define some of the underlying elements required for security metrics and how security managers can use them to quantify different portions of their security programs. We can't provide all the answers, but we'll provide you with a better understanding of the complexity of security metrics and insight into how you can leverage existing data to strengthen performance measurements.

Calculating Asset Value
Enterprises routinely place values on all information assets (hardware, software and data) that are reflected in IT spending. Enterprises expect every hardware and software installation to return an amount that's at least equal to its total cost of ownership (TCO) over its life expectancy.

If we accept that premise, we can assume that the minimum value of all computing assets is the amount of IT spending for a year (salaries, operations and maintenance) plus the depreciation or amortization value of the assets (hardware and software). An ERP system with a TCO of \$20 million must return at least that much over its lifespan.

But such calculations need context. That's why we assign quantifiable values to information assets for objective evaluation and comparison. The following are some ways to classify your information assets' value:

Productivity Value. An asset's worth is at least as much as the costs of implementing, maintaining and using it. For a single PC, the minimum information asset value is the cost of the PC plus software, a percentage of IT overhead costs (e.g., help desk) and the user's time (salary). As you'll see later, productivity value is essential for calculating other security metrics.

Revenue Value. For some assets, worth is measured in the value of transactions. If your e-commerce Web server processes \$1 million in transactions a day, it's worth \$365 million annually.

Revenue value isn't always clear-cut. Supply-chain systems, such as manufacturing equipment and control systems, don't generate revenue, but your revenue generators are useless without their productivity. Their value is often calculated by measuring how much revenue would be lost if those systems were unavailable.

What isn't as easy to calculate is the revenue value of a pure information asset, such as software, music, movies and electronic documents, that can be perfectly replicated. The assets' values are usually much higher than their assigned prices, since they're sold many times over. The value of an "information asset for sale" can be estimated using historical data and/or revenue trend information.

Liquid Financial Assets Value. Those much-vaunted "assets under management" figures associated with financial institutions provide a straightforward way to assess their value. If \$1 billion is under management, that amount, plus the productivity value, provides the total value that's being protected. Add to that the revenue values of transaction assets, and you've got the total value of information assets that require protection.

Intellectual Property Value. This is the most difficult asset to value. It's usually seen as "the reason" a company is in business. There are books with elaborate formulas to calculate intellectual property value; however, it may be easiest to consider intellectual property's contribution to a company's market capitalization. This value can be calculated by multiplying the amount of intellectual property captured on systems with the difference between market capitalization and book value.

Calculating Potential Loss
Calculating potential losses is a little different than measuring asset value. Information assets aren't usually "lost" in the traditional sense--they're often still available for use, and partial (or even greater) value may be lost. Some information, such as Social Security numbers, don't have an inherent value; music files that are sold multiple times will have a greater asset value than face value because of their potential revenue. Value is a matter of context.

Because asset value is linked, but not tied, directly to loss, you must consider the type of compromise when evaluating potential losses. There are five distinct types:

Confidentiality breaches occur when high-value information, such as trade secrets or financial data, is read from storage, sniffed from the network by unauthorized parties, or leaked by internal users.

Integrity breaches occur when data is modified in transit or at rest, such as with transactions that are modified to reflect inappropriate quantities or dollar values.

Availability breaches are when information is deleted or made unavailable.

Productivity breaches occur when resources are disabled, such as when an e-mail server is disabled by spam or an e-commerce service is DOS'd by transaction requests.

Liability breaches occur when systems and data are still available, but are being misused. This could be an employee using corporate resources to host a porn site or a hacker storing stolen files on a compromised server.

Determining information asset value enables enterprises to focus on their real security needs and allocate adequate resources. But asset value isn't the only thing that may be lost. It's important to consider incident costs with the lost asset value when evaluating actual and potential losses. Incident costs are the lost asset value that are specifically associated with the incident, including IT productivity, legal and regulatory costs.

The severity of an incident and its losses are calculated by correlating the type of breach with the value type. For instance, if a worm disables your network, you have an availability and productivity breach that affects the asset values of everything on the network--productivity and revenue. The loss equals productivity and revenue values, plus incident costs associated with IT productivity. Legal costs and regulatory fines are also a factor.

A popular method of calculating potential losses is annual loss expectancy (ALE), an estimate for expected loss calculated by multiplying the probability of a particular type of loss by the total loss potential. For example, if there's a 10 percent chance of losing \$1 million, the ALE is \$100,000.

Obviously, there's a difference between a 1-in-10 chance of large loss and the strong likelihood of a number of incidents throughout the year. One way to address this difference is by treating ALE based on the category: recurring losses or specific security breaches. These distinctions allow you to rate the viability of projections based on historical data. If you know you've suffered an average 100 malware infections a month (recurring), you know your projections have a greater validity. Predicting less-frequent attacks is much more ambiguous, making such calculations more art than science.

ROI: Positive Returns
 Return on investment (ROI), from a business perspective, is elementary: You spend money with the expectation that you'll make back the same amount, plus some revenue gain. An e-commerce server that costs \$20,000 and processes \$100,000 in transactions will have a \$80,000 ROI, or 400 percent. Security is a cost center like human resources, the mailroom and facilities; it doesn't generate revenue. Even here, though, evaluating ROI is straightforward. You compare actual costs--both known spending and known incident costs--with projected costs, based on assumptions and expectations about what will change with the new product or service. The ROI is savings generated by efficiency. An automated patch management solution, for example, will reduce the time it takes to deploy patches compared to a manual process, resulting in a cost savings. The same logic is applicable to most any security process: services vs. in-house operations, or help desk vs. manual password resets, IPS and incident reductions. If the savings is greater than the product's TCO, you have positive ROI. Generally speaking, ROI is gained by increasing the productivity of your IT staff through automation and reducing recurring, predictable incident costs. Frequent incidents and their associated response costs are a factor in some security spending, such as antivirus and antispam, in which ROI is an evaluation of operating costs under different conditions, such as running a network with and without AV. --Pete Lindstrom

Measuring Security Spending
As part of the overall security management program, it's important to measure total spending and spending effectiveness. Measuring enterprise-wide security spending is more difficult than it appears. Security spending is often divided among various business units and departments, as well as being lumped in with network and infrastructure spending. Finding all of the security dollars and separating them from other budget items is a daunting task.

Sort through the IT chaos using activity-based management by classifying different forms of security activity and correlating all spending related to them. An example of this is the "Four Disciplines of Security Management" (www.spiresecurity.com/fourdisciplines.asp): trust management (policy enforcement, security architecture, certificate management), identity management (user account management, password resets), vulnerability management (configuration hardening, vulnerability remediation) and threat management (monitoring, threat analysis, incident response, forensics). The classification system doesn't matter, as long as you're consistent in your application.

What spending information do you collect for your activity-based analysis? Everything. Spending is broken down into salaries (the percentage of people's time spent on security), hardware and software, and support and maintenance costs.

Staff (people) cost is a matter of identifying wage data for the portion of time anyone spends on security activities. The goal is to identify the percent of time staff members spend on security and calculate it as a percentage of their salaries. This is similar to calculating productivity. After doing this for every individual, the aggregate amount is the total enterprise security spending for personnel.

Allocate these amounts across each of the security activities in the model. The easiest way to do this is to allocate time by percentage for each activity. The percentages can be used to calculate spending for each activity by employee, department or the entire enterprise.

Finally, collect spending for products related to the security activities. The TCO includes product costs, and maintenance and support fees. Dividing TCO by the product's life expectancy is useful to annualize the costs (this is similar to how accountants depreciate and amortize costs over a number of years).

The result is a clear picture of how much an enterprise is spending on security, which can then be broken down by department for comparative purposes. Over time, activity-based analysis will be able to show trends in security spending. And, it's a good method for evaluating security purchases and calculating return on investment (ROI) (see "Positive Returns").

Measuring and Reducing Risk
Traditionally, security managers estimate risk (the probability of something bad happening) as a percentage incorporated into the ALE formula. If the expected loss from a compromise is \$100,000, and you estimate that it could happen once every 10 years, the risk is 1 in 10 and the ALE is \$10,000.

But which do you pay more attention to: the 10 percent chance of losing \$10,000, or the 1 in 100 chance that you'll lose \$100,000? Again, risk is often like art--judged by the eye of the beholder.

Defining risk for an entire enterprise is next to impossible. A more manageable approach is breaking it down into small units and types. There are three common forms of risk:

• Manifest risk is the ratio of malicious events to total events in a computing environment. There are at least four event types: flows, sessions, program commands and transactions. You calculate manifest risk by counting the number of flows and either estimating or using historical data to count the number of malicious events.
• Inherent risk is the likelihood that system configurations will contribute to a compromise. This is calculated by counting a system's configuration attributes like open ports and services running, and comparing the number of associated vulnerabilities.
• Contributory risk is a measure of process errors or mistakes made during the normal course of operations and maintenance that contribute to a compromise. Errors associated with user account management procedures and vulnerability management procedures are fertile ground for measuring contributory risk.
Even with these three risk types, you still need to determine the security event/incident frequency. First, you can use historical data to predict the future. If you can identify, for example, the volume of malicious traffic using IDS logs and have already counted the total number of flows, you can project future risk.

Obviously, it will take some time before these numbers are available, and some may never be. But there are other benefits: Even without a number for malicious events, it's possible to measure an environment's exposure.

Exposure is about possibility rather than probability (risk). Exposure measurements can be used as a relative comparison within an environment or across companies. If one can assume that risk is constant for like-sized companies (even if we don't know the number itself), this exposure measure can act as a "risk proxy" to measure the relative difference in risk levels.

Given the elusiveness of quantifiable risk, exposure can provide a means to measure your effectiveness at reducing risk.

None of this is designed to answer all your questions about metrics. But, methodologies outlined for calculating asset value, potential loss, security spending and risk give you a foundation for gathering data and applying it to your own goals and expectations. And, it will bring you closer to assigning values to that security formula and defining what risk means to your enterprise.

This was first published in February 2005

Content

Find more PRO+ content and other member only offers, here.

Oldest

SearchCloudSecurity

• A look at the cloud migration challenges enterprises could face

One of the most common problems facing organizations is dealing with many cloud migration challenges. Expert Dave Shackleford ...

• Adding a cloud security policy to your overall IT defense strategy

Enterprises should consider a cloud security policy in the wake of more services moving to the cloud. Expert Mike Chapple ...

• Microsoft wins email privacy court battle with U.S. government

The U.S. Court of Appeals ruled in favor of Microsoft regarding a controversial email privacy case with the U.S. government that ...

SearchNetworking

• Cisco vCenter plug-in bridges physical, virtual data center networks

Cisco has launched a vCenter plug-in for connecting the VMware management tool to applications running on non-virtualized servers.

• ONOS-based open switches power leaf-spine fabric for cloud, carriers

The ONOS project has introduced a Layer 2/3 leaf-spine fabric comprised of the group's operating system and Edgecore Networks' ...

• Taking your data center network design to the next level

Data center network design upgrades are now high on many enterprise's must-do lists. The way enterprises now use data is why: In ...

SearchCIO

• CIO, business must wield EU-U.S. Privacy Shield

CIOs have a part to play in adopting the enhanced data transfer pact. Also in Searchlight: Survey says majority of companies not ...

• 'Promiscuous' users redefine bounds of good mobile security

To meet today's mobility demands -- and deliver good mobile security -- organizations must balance business and user needs, says ...

SearchConsumerization

• Android, Windows tablets from HP take aim at business users

HP released a new line of tablets targeting business users. The HP Pro Slate 8 and Pro Slate 12 run Android and cost \$449 and ...

• Microsoft to lay off 18,000, Nokia X moves to Windows Phone

Microsoft will lay off 18,000 people over the next year while the Nokia X line of Android smartphones, which was unveiled earlier...

• Microsoft Surface Pro 3 vs. Microsoft Surface Pro 2

Surface Pro 2 and Surface Pro 3 are different enough that Microsoft is keeping both on the market as competing products. Which ...

SearchEnterpriseDesktop

• Internet Explorer support for old versions is over. What now?

Microsoft stopped supporting Internet Explorer prior to version 11, but the web browser is still present in many organizations. ...

• Navigate a maze of installers with PackageManagement

PackageManagement -- formerly OneGet -- lets desktop admins use PowerShell cmdlets to work with several different installer ...

Admins who still have Windows XP systems on their networks can turn to a Windows POSReady 2009 registry hack to reopen the update...

SearchCloudComputing

• Data privacy in the spotlight with Privacy Shield, Microsoft

Data privacy continues to be a hot-button issue on both sides of the Atlantic, with the Privacy Shield agreement and a big win ...

• Build a cloud redundancy strategy using multiple providers

No cloud platform is perfect; there is always a risk of an outage or data loss. Consider a multicloud model to ensure redundancy ...

• Tapping next-gen app markets, DigitalOcean adds block storage

Customers seeking to scale out memory independent of compute resources are the focus of a block storage addition from cloud ...

ComputerWeekly

• No barrier to trial in UK Lauri Love extradition hearing told

British hackers, including members of Anonymous, have been tried in the UK, despite causing serious damage to US computer systems...

• Yahoo confirms nearly \$5bn sale deal with Verizon

Verizon Communications to boost its advertising business by acquiring Yahoo's client database in a \$4.83bn deal

• BT says it's "absolutely willing" to create independent board to run Openreach

BT Group chairman Michael Rake says the organisation is willing to let an independent board oversee Openreach’s spending and ...

Close