Gartner’s Ramon Krikken on Web application security scanners

Web application security scanners aren’t a new technology, but enterprises often fail to make the most of them.

In this interview from the 2011 Gartner Security & Risk Management Summit, Gartner IT1 Research Director Ramon Krikken discusses why successful use of Web application security scanners depends on satisfying the needs of information security and development teams. Plus, Krikken talks about the four key issues to address when considering a Web application security scanning implementation, and details the state of the product market and recent Web application security scanning technology advancements.

Read the full text transcript from this video below. Please note the full transcript is for reference only and may include limited inaccuracies. To suggest a transcript correction, contact  

Gartner’s Ramon Krikken on Web application security scanners

Eric Parizo: Hi, I'm Eric Parizo with Thanks for watching;it's great to have you with us. Joining me today is Ramon Krikken, research director for Gartner IT1. Ramon, thanks so much for being with us today.

Ramon Krikken: You're very welcome.

Eric Parizo: Ramon, you've written that successful Web application security
scanning implementations have to meet the needs of both an enterprise’s
information security team and its developers. Why is that?

Ramon Krikken: Mostly because like any other activity that we consider part of
the application security program, it is something that should be an
integral part of the software development and acquisition life cycle. When
you look at the number of applications, especially that enterprises have,
how quickly things turn around, the size and complexity of these
applications. If you imagine that the security team were the ones to always
have to do all the scanning, to always have to work with all the results,
to always have to do this, to always have to do that, it can quickly get
out of hand. It doesn't really scale all that well. The security team
would be, in the end, a good resource to help out, to define policies, to
come up with new ways of doing things, new attacks, new ways of scanning.
But if you think about how software is developed and acquired, it would
make a lot of sense, at least eventually, to say, "Well, maybe it is the
development team that should do some of these day-to-day tasks."

So, that is in the future. Today though even, these development teams have
to be able to work with the results from these scanners, right? So even if
it's the security team that performs the scan, somebody needs to work with
this report and say, "OK now, what does this mean for how we need to fix
the software?" And what we found, I think, over the history of security
products is that they're usually by security people, for security people.
And so, when you take the output of any kind of assessment tool, it's
usually not quite that understandable for anybody other than a security
expert. And so, when we take all those factors into account, I think it is
pretty important that we look through both lenses and say, "Can this tool
or this service provide the things that both of these teams need?"

Eric Parizo: So with that said, how do you do that? How do you find that meeting of
the minds?

Ramon Krikken: Yeah, that is not an easy one. We recently completed some field research
on what has SDLC basically. And it was kind of surprising how few times
security came up. So, we still see, and this is through talking to
vendors, through talking to customers, there is this split between the
developers and the security team, and everybody's kind of doing their own
thing. And what happens there, of course, is that the development team is
told, "You need to develop product X with Y functionality within line Z."
So I'm with the security team, and I come along and say, "Oh, you need to
do security, as well. Here's a tool or here's a scan report. Now go fix
it." How are they going to be able to get that done and still meet their
deadline? So, there's a lot of pre-work that goes on when you look at
these things in terms of making everybody comfortable with the fact that
this is now something we have to do. And then that goes for the entire
program. So usually what we see is that Web scanning kind of starts up as
an assessment tool. The security team just runs it. They take the report.
They then do some reading through it, see what it means, and see if there
are false positives and scratch them out, and then say, "Well, we think
that this means that you, the development team, need to do the following
things in order to fix it." And there are several ways in which you could
use a Web scanning tool. The assessment is one end of the spectrum,
completely integrated with day-to-day quality assurance, and testing is
another. Kind of picking and choosing where it goes to begin with, and
then see if you can work back into the development life cycle, is going to
make a lot of sense to do it that way. And it's particularly because in
the beginning, when you start Web scanning, it might very well be that you
don't even know what is wrong with the software. It might be the first
thing that you are doing with certain applications. So if you don't even
know what you're going to find, it's a little premature to make decisions
there. But ultimately, unfortunately, it's much of a "process in practice"
question. Again, because the tools aren't necessarily straightforward to
use – sometimes for anyone, but in general only usable for the security
team. Kind of working together, making sure that you can translate from
test cases into the test and then back to what needs to be fixed. That's
usually how it's done.

Eric Parizo: You've written about four key considerations for Web application
security scanning implementation. Applicability to the software
environment, accuracy in finding flaws, alignment with development security
processes, again, and actionability of results. Tell me why each is so

Ramon Krikken: So let me start with actionability of results because even though
it's last, it is one of the most important aspects when we look at fixing
software because that's ultimately what we talk about. So we can say that
Web scanning is all about finding vulnerabilities. But, that would be kind
of narrow-minded. It would really take a security lens there. It's like:
"Let's find what's wrong." But in the end what we need to do is say, "OK,
how can we fix this. What do we need to do? What do we need to do to this
specific piece of software, in order to make things better. OR, or 'and',
later on, What do we need to do to the software development process in
order to reduce the number of times this occurs in the future?" So if you
look at some of the outputs from Web scanning tools and also from static
analysis tools, it doesn't really matter what you use. You see that it's
very problem focused. It kind of makes sense; it is a problem finder, you
get problems as output. But, if the tool doesn't really provide you the
detailed information that you need in order to pinpoint where the problem
is, what it looks like, on what circumstances it might be triggered, it
could be pretty difficult to figure out, "OK, now what do we need to do to
fix it?"

And we're seeing some advances there by vendors in terms of how they
present results to people. So it used to be, it was really just an output
about stuff. Now we actually see some remediation advice perhaps. Or,
starting out as general remediation advice, now maybe you can plug in your
own advice. So your developers may do job volume, may be able to say,
"OK, this is how you need to do it in this language." Hopefully in the
future, there might be standards around how you do these things, and then
you can say, "OK, well, we found an SQL injection vulnerability, so you've
obviously not used the enterprise standard library that tries to prevent
this stuff." So the output, and how we use the output in the end is going
to be one of the most important things.

We already talked about alignment with the SDLC. It just means that
regardless of where in the life cycle you put it, you're able to do it in
such a way that you get the most out of the tool or the service. So, very
early on in the SDLC, when code barely is developed and it doesn't even
run, it doesn't make much sense to use Web scanning because there's
nothing to test yet. That's for example, where static analysis might go.
And then as you move on, and the product becomes more functional, that's
where you can really put it in. One thing we didn't talk about is actually
to use ongoing use of Web scanning in production applications. So we could
say, "Wait, once it's in production, what do I care? It doesn't really
change, so why do I keep rescanning it?" But there are two things about
it; one thing we might find is that when an application goes into
production, the production environment doesn't really mirror the quality
assurance environment sometimes. So here everything tests fine. When you
put it into production, all of the sudden all sorts of problems pop up. So
that might be one reason to do it.

The second is that some things may actually not be dependent on the code
but something in the back end; maybe a server configuration that turns a
certain filter on or off, if you're not constantly monitoring for these
things, you might miss the fact that something got upgraded or changed or
reconfigured. And so that's why aligning the different capabilities with
the SDLC is going to be important.

The accuracy, now that's always a bit of a sticking point. Testing has
shown to some extent, that automated tools are only partially effective at
finding problems, no matter what tool you use. The NSA recently redid
their evaluation of static analysis tools for example. The results were
pretty bad. The tools didn't find as much as they had hoped they would.
Less have been available for Web scanning. Most of the tests have been
more comparative tests to see what scanner does it better than another.
But after the customer, and now it's a little while ago, did an evaluation
of Web scanners versus static analyzers versus what the consultant can
find, for example. And the Web scanners didn't fare all that well. They
found only a very small percentage of the vulnerabilities that they knew
were in there, because they had put them in on purpose. But what you do
get, and it's kind of disappointing, you kind of want them to find more
than 10% or 15% or 20%, but the big thing to realize there
is that what they do find are things that other people would easily find as
well. So they find those holes that you can drive a truck through. And if
you're not even looking for those, you're definitely missing something.

On top of which, the different types of scanning and validation don't quite
overlap in what they find. So static analysis finds some problems that Web
scanning doesn't find, and the other way around. So you can't really say,
"Static analysis is on average more accurate so we'll just use that."
Because then you're still missing things. So it kind of comes down to
realizing the limitations and knowing that you're going to have to do some
additional tasks above and beyond - perhaps penetration testing. Have a
consultant look at the code if it's particularly critical. So that is why
that matters and I think the most important take away is to just realize
that the tools are not perfect.

And finally, you really need to be able to work with the tool pretty well.
Alignment with the environment really comes down to making sure that the
scanner can work with the software that you have. So if you look at a lot
of the software security products out there, they're geared on the
traditional three-tiered application: presentation, application, database.
It's very evident in some of the scanning templates. It's got an SQL
injection line, it's got a cross site scripting line, it's got a cross site
request forgery line. But what if you use service-oriented architecture?
What if you use an XML database? What if this is some application with
very convoluted business logic that really requires a lot of knowledge of
how you click through the pages. How do you make the scanner work? And
that is where some challenges come in. Even the scanning vendors will say,
"Yes, we support some Flash for example, but if you use other rich
internet application frameworks, it might not." A lot of them support
Javascript, but there may be certain constructs that don't execute well.
And on top of that, to go back to the business logic, it is quite difficult
to have an automated scanner fully understand the application sometimes.
Let's say you have to go through a sequence of screens and screen number
one says, "Input your email address." Screen number two says, "Input your
date of birth." Screen number three says, "Now input your name as first
name, comma, last name." You can't really tell from just looking at the
fields unless you understand the text that is there that you might need to
format your data a certain way. And so the scanner might just get stuck.
It gets sent back to the first page and might give up. And so it's not
trivial to get some of these scanners to work with modern applications. And
that's really where that comes in. It's just making sure that it can work
with what your enterprise portfolio looks like.

Eric Parizo: All right, let's wrap things up, Ramon. Give us a quick snapshot of
the market. How mature is it today? What are the key capabilities that
are out there? In terms of addressing some of the issues you and I just
discussed, where are the products in terms of addressing them?

Ramon Krikken: So the market has been fairly mature for a while now. Web scanners
have been around for a long time. Now some of the vendors have been bought
up by larger vendors and they are being integrated into their larger
software security or application development portfolios. Yes, there are
advancements that are being made that I think are quite interesting on
several of these topics. Mainly - so, we've already talked about more
usable results. That's one thing. Everybody's looking at that because
ultimately they realize, "We need to sell this to people, and they need to be
able to use it."

Another one would be being able to automatically load the results from the
Web scan into something like a Web application firewall. So you don't have
to fix the code right away, but you may be able to load the signature into
something to block it. But two other things that are getting more attention,
I think, are the idea that you can walk the scanner through the
interface. And then it will take your input, and kind of then do its own
thing. It's been present in products for a little while now but I think
it's getting more mature and more usable.

And then finally, this idea of combining the Web scanning with other
components. So a hybrid analysis - not just doing it in isolation but
putting things together. That is getting a lot more attention. Although I
think a little bit more research and proving of the solution is required
there. But I do think that that is going to help a lot, particularly
because again, if these scanners only find 20%-25% of the problems that are
there, anything that we can do to make that better of course is helpful.

Eric Parizo: Ramon Krikken, research director with Gartner IT1, thank you.

Ramon Krikken: Thank you.

Eric Parizo: And thank you for joining us as well. For more information security
videos, don't forget you can always check out I'm Eric
Parizo. Stay safe out there.

View All Videos

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.