This content is part of the Essential Guide: How to develop software the secure, Gary McGraw way

Gary McGraw on secure software development

For Gary McGraw, the chief technology officer of Cigital Inc., software security is not just an item to check off on a list. It is a personal crusade, an effort to educate developers, QA testers, senior management and anyone else with authority over a software project about the value of implementing secure software development practices and the effect they can have on a company's bottom line down the road. In this interview with Executive Editor Dennis Fisher, McGraw discusses the progress the industry is making on software security and how much farther most vendors have to go.

Read the full transcript from this video below:  Gary McGraw on secure software development

Dennis Fisher: Hi, and thanks for joining us. I'm Dennis Fisher, the Executive Editor of the Security Media Group at TechTarget, and joining me today is Gary McGraw, the CTO of Cigital. Gary, thanks for joining us.

Gary McGraw: Thanks.

Dennis Fisher: Let's start off by talking a little bit about where things stand sort of in the state of software security in the industry these days. Are you optimistic or a little bit pessimistic about how things are right now?

Gary: McGraw: I'm absolutely optimistic about the progress we've made, especially in the last five years, and I wrote a book on software security in 1999, and at that time, I couldn't even sell the idea to my Mom, right, and now everybody is talking about building security, and they even say it in the key notes at RSA. You hear the message being spouted all over the place.

The interesting thing that happened about three years ago all over the place, was that people started transitioning from talking about the problem, from philosophizing about it, to actually doing something about it, so almost every major firm on Wall Street has done a lot of software security initiative work, much of which is more impressive in ways than what Microsoft has done. They just haven't been making a lot of noise about it, but we made a huge amount of progress, both in training tens of thousands of developers in finding the right sorts of tools for testing people, black box tools for testing, and white box tools for developer types, and in setting up most importantly, governance and metric systems for executives so that they can find out if the money that they're pouring on this problem is actually solving the problem or not.

The good news is that it is, and so what I see happening on Wall Street is beginning to happen to the banks on the West Coast, it's spreading over this way, and the middle market is beginning to pay attention to software security, so I'm hugely optimistic about the progress that we've made in the last five years, and especially the last two years. I think the market has doubled almost since two years ago. It was about $180 million, last year I think it was maybe $250, and this year maybe $500 million market. That's a real market and that means that people are buying services, they're buying tools, they're actually trying to solve the problem.

Dennis Fisher: I've heard a lot of people say that solving the software security problem is going to… it's very expensive. It's going to cost them a lot of time and money in the development process. Is that true? Does it really add that much more cost and time into the development process?

Gary McGraw: It's true if you look at the wrong number, so if you measure software costs starting from requirements only to when you ship software, that accounts for 30% of actual total cost of ownership of software. That is, 70% of software costs happens after you've shipped the software, to do things like fix bugs and do revisions and stuff like that. Now if you understand that you need to think about the whole number, the whole software number, and you manage to that, then what happens is the total cost of ownership goes down if that first little component of the cost of 30% up front dev cost gets a little bit bigger.

If you're only looking at that little number, then you're managing the wrong way. What happens is, if you make that number go down and down and down, quality goes down and down and down too. The other number, if you're not tracking it, is going towards infinity real fast, and that's the problem. So those software management people and risk management people that are managing to the right number, doing risk management over the total cost of ownership software, are finding huge returns.

And one of the people who is doing a great job is Phil Venable, so now everybody knows that Phil is doing astoundingly good things in security. This is one of the good things that he's done to push software security into the development life cycle, so when people say, "Oh, it's too expensive to do it right," you would say, "Well, let's look at the right number," and then you find the right answer.

Dennis Fisher: I know there's a lot of training that goes on in the professional world in terms of software security for developers, but is there any more of that that's going on in colleges and universities right now, than there was say five years ago?

Gary McGraw: I don't think so and I don't really hold out much hope in pushing that into the curriculum because computer science is kind of a screwed up discipline anyway. So what are you not going to teach? Compilers? Operating systems? And we barely teach them to code today, much less try to teach them to code in a secure manner. I would like to see that happen, I just don't really see that as realistic, and anybody who really hires programmers like Morgan Stanley with their 10,000 guys that write code, they're not looking at universities to produce professional coders for them. They're taking people that have years of experience either outside of school or stuff they've done since they were 15. The best coders, they just have done it all their lives. I started coding when I was 15 years old. That's a long time ago now for those of you listening, and I don't think it's something that you really teach in school.

Now, on the other hand, training developers that are already quite good developers, about security, is a very important thing. We started doing that in 2001, actually at Goldman Sachs. Since then, I've probably trained, well my company has trained about 10,000 people I'd say, more or less, and that's just a little tiny small sliver of the number of developers out there in the world. There are many more people that need to be made aware of the issues at all, and it's just as important, I want to emphasize this, to focus attention on architecture as on bugs. The problem with a lot of software security training today is it amounts to the bug parade, so we talk about cross-site scripting, sequel injection, buffer overflows, blah, blah, bug, bug, bug, and it's important to eradicate those bugs, but half of the defects are architectural in nature.

So enlightened executives like Justin from Omgeo, who used to be a State Street, have focused more attention on architecture when they're doing their training, and that's important. Don't just stop with introduction to security and check the box and assume everything is all kosher. Focus some attention on architects, focus some attention on testing people, and most importantly of all, senior executives and middle management, especially product managers, need to have a clue about software security. If you do that in a coordinated fashion, you'll make a huge amount of progress.

Dennis Fisher: What about the commercial software vendors, how much progress are they making on this problem? I know Microsoft has done a lot, Oracle, maybe Cisco, some other people, but what about the rest of the industry? Is anybody else making progress on software security right now?

Gary McGraw: So the Cisco guys also have a program, at least on the product side of the house. Some software vendors have made progress on that. I think that the software vendors, with the exception of Microsoft, are really behind the Wall Street firms, and so I've seen more progress in those firms that are concerned with financial risk management and deeply understand risk management, pushing software security along, than any software vendor. Now, one of the biggest ironies of all is that security vendors in particular, are particularly bad at software security, because they think, "Well, we make security products, so we must know how to do software security." Turns out that's wrong, so whatever we can do to put pressure on all software vendors to think about software security is important.

One of the trends in the market that we've seen at Cigital which is really cool is some of our customers asking their suppliers to get their act together from a software security perspective. So one big vendor will say to another vendor, "If you'd like you X gillion dollars next quarter, you're going to need to show some evidence that you've done software security." And they go, "What's that?," then they say, "Well talk to Cigital, we'll figure it out." I think that's a very nice thing to see happening. We've seen it more and more in the market, so that's good. That means the market is beginning to work properly.

Dennis Fisher: Can the enterprise, the guys that are actually buying software, put some pressure on the vendors as well through purchasing decisions...

Gary McGraw: Absolutely, the Verigo guys talk a good game when it comes to this, and it's kind of that phenomenon that we talked about before. If you're buying stuff from someone and you're going to buy a lot of it, then you often negotiate the contract in special ways. You can add security things into that contract, both for COTs and for Bespoke software. I've seen it work for, NSD was buying some stuff, sorry, scratch that, can't say that. So a stock exchange company was buying some stuff from a vendor and wrote the service level agreement in such a way that security played a really important role, and turned out saving millions of dollars because the software wasn't up to par from a security perspective. It had to be fixed and the vendor had to do that on their nickel.

Dennis Fisher: Are there one or two problems that really worry you in security software right now, things that keep you up at night?

Gary McGraw: Number one is an architectural issue and it has to do with Web 2.0, software as a service, Web 3.0. The whole idea is this, you have a massively distributed system and part of your system is actually outside of the trust boundary. For example, it's running on the customer's PC, why should you trust the customer, or in the terms of on-line games which I've just written a book about, as you know, Warcraft is set up to be a massively distributed system where 10 million people run part of the game on their PC and you expect them not to cheat? We have this problem of sharing state in a massive system across trust boundaries over time, and that's a very difficult challenge.

Most software architects don't think about trust boundaries and they have a hard time thinking about what's called re-entrant code. In a multi-threaded environment, they're still at the stage of just getting their stuff to work, much less work in a hugely loaded environment that's being lagged by a bot net, in order to cause this sector of the net to misbehave. People just don't think about those things when they're developing systems, and I think that the more progress that we make towards the good aspects of Web 2.0 and Web 3.0 and SOA and all these things, the more we move towards real massively distributed systems, the more that's going to be the number one issue. I would love to stop talking about cross-site scripting, yawn, and buffer overflows, get a new language, or sequel injection, come on, let's get real.

Dennis Fisher: Gary, thanks for joining us today. I appreciate it, and thank you all for watching. That's all the time we have today. If you want more information on software security, go to Thank you.

View All Videos

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.