Manage Learn to apply best practices and optimize your operations.

Gary McGraw: When risk management goes bad

Expert Gary McGraw suggests simple guidelines for risk analysis and risk management using software risk analysis as an example.

Everyone knows that risk management is the mature way to think through information security. That's because a careful balance must be struck between security on the one hand and business functionality and efficiency on the other. Too much security can ruin a good business proposition by bogging it down in a security swamp. But too little security can lead to unacceptable levels of risk that, if realized, can threaten the very business itself. Like any executive, a CISO's job is to make risk management decisions and communicate those decisions effectively to the rest of the executive team and to the company. To preserve job longevity, these decisions must be in line with corporate culture, compliance and regulatory mandates, and "community standards" of acceptable diligence. They must also be technically sound.

The problem is that risk management is non-trivial -- and sometimes it goes bad. Without throwing the risk management baby out with the bathwater, let's explore what happens when risk management goes bad in two failure conditions. I'll approach this exercise through software security and software security portfolio management just because that is what I'm most familiar with.

Failure 0: Too much of your portfolio is 'naked'

Imagine security controls as clothing -- the kind of clothing that protects your naked body from the elements. Further imagine that you have a large portfolio of "naked" software assets to protect. Needless to say, you have a limited budget to spend on clothes. The risk management problem is one of determining how much of what kind of clothing to put on which software assets.

Gary McGraw logo

One obvious idea is to divide the portfolio into categories of risk. Probably the easiest and the best way to do this uses three simple categories: high, medium and low. This division can be accomplished by tagging various software assets with categories like: Internet-facing, directly-controls PII, handles money, life threatening, schedules conference rooms, etc. Then you put each software asset into one of the buckets according to the way you weigh each of its category tags (e.g., by thinking through the negative case of each process or outcome not happening as planned).

The question is then what to do about imposing security controls on each of the buckets. Just as one flavor of example (testing in this case), controls can include things like: architecture risk analysis, code review, dynamic analysis, penetration testing, and so on. So, maybe we decide to allocate our scarce resources such that we apply all controls in our power to the "H"bucket, a handful to the "M" bucket, and we ignore the "L" bucket entirely.

In fact, the kinds of risk management decisions you make for the buckets makes all the difference between risk management reasonableness and risk management fail.

Consider Figure 1. In this example, only "H" bucket assets have any security controls imposed, and those are imposed linearly according to perceived risk inside the high bucket. The problem is that no "M" or "L" assets are even considered for controls. Frankly, that's really bad.

Example of badly managed controls.
Figure 1

If a majority of your portfolio falls to the "do nothing" line (in our example that would be the x-axis), you probably have a problem. (For instance, think through the case of a retailer whose software assets include shared access by an HVAC supplier and you'll see what I mean. Low risk? Hmm.) One simple way to solve this problem is to come up with some easy-to-instantiate security controls that can be applied across the board to every software asset.

To be flip about it, none of your assets should be entirely naked out there, they all need underwear at least! So find at least some minimal security control equivalent of underwear and hand it out. What might make a good example? If there are three knobs on security testing (number of apps, number of tests, and types of tests) and we're setting knob one to "all," we can set the other two knobs to appropriate levels, rather than leaving them at zero. Maybe we do a black box dynamic test on every asset, or maybe something slightly more effective, but "nothing" is not an option. Indeed, I think the day has arrived when the FTC, states' attorneys general, cyberinsurance companies, shareholders, and popular opinion all agree on this point.

We want to accomplish two things in terms of Figure 1. We want to move the security control minimum line "up" and we want to flatten the curve by bending the low end up as well. That will result in a pattern looking more like Figure 2. (Once again, just to be clear controls in both the "M" and "H" buckets are applied linearly according to risk inside that bucket.)

Better management means low and medium risks aren't uncovered.
Figure 2

Failure 1: How many mediums make a high?

In software, there is always a bug severity problem. How many little bugs does it take to equal a big bug? In some cases, too many niggling little bugs make code smell so bad that nobody wants to use it. No individual bug is a show-stopper by itself, but taken together the show performs to an empty house. This bugaboo of a problem has persisted for years.

The problem of compounding applies in spades to security risk and risk management decisions. How many low risk or medium risk software assets can I stomach at what level? What is the real truth behind classifying something as "medium" risk? Does risk compound like bugs seem to? Can your portfolio have a smell like bug-infested code does?

You can probably divine some kind of answer from the shape of your software asset risk curve. If your buckets are properly configured, your assets will likely fall into a standard distribution -- and end up looking like a Bell curve (see Figure 3).

Normal distribution of software asset risk is a bell curve.
Figure 3

But they may also fall into a shape like the one in Figure 4 if your categorization scheme is not good enough or if the nature of risk in your business is a special snowflake.

One example of abnormal distribution of software asset risk.
Figure 4

In that case, the sheer number of medium risk assets calls out for some kind of uniform set of security controls to be applied. Relatively few highs means that a high-only strategy makes little sense. In fact, I believe that in no case at all should a high-only strategy be adopted!

The same lessons we learned for Failure 0 also apply in the case of Failure 1. Move the security control minimum line "up" and flatten the security control delta between high and low as much as possible. It would certainly be a bad choice to artificially manipulate the definitions of low, medium and high to simply hide the problem. You might as well set your bathroom scale back 20 pounds and call your New Year's resolution a success. Manipulating the definitions to make everything high is just as bad as in that case management won't buy your FUD.

Risk management über alles

Ultimately, both risk management failure conditions I highlight in this article can be addressed by the same guiding principles: leave no code naked; tighten the security control delta between low and high as much as possible without moving high down; create a specific testing schedule that ensures full portfolio coverage on some reasonable schedule, even if it's once every couple of years for internal low-risk applications (seems like a long time, but it's infinitely shorter than the "never" that happens all too often now).

Next Steps

Learn more about using your own risk management framework.

This was last published in February 2015

Dig Deeper on Secure software development