Problem solve Get help with specific problems with your technologies, process and projects.

Open source testing tools for Web applications: Website vulnerability scanner and recon tools

Google’s open source testing tools for Web applications can save organizations money and improve the security of Web apps.

Can you describe how a Web application security reconnaissance tool such as Google’s Skipfish, Ratproxy and its...

newly released DOM Snitch works? What role can these tools play in an enterprise Web application security testing program? Can they, in some instances, take the place of commercial tools?

The code behind today’s Web applications, even those that appear to provide a simple service, is becoming increasingly sophisticated and complex. This complexity inevitably leads to an increase in an application’s attack surface and, in turn, a higher likelihood of coding flaws creating potential vulnerabilities. As part of its contribution to the information security community, Google Inc. has made available various open source tools, including the three you mention. Developed by Google’s information security engineering team, these tools are also used internally at Google.

Skipfish is a Web application security reconnaissance tool or, more simply, a website vulnerability scanner. It works by carrying out a recursive crawl combined with dictionary-based probes to generate an interactive site map of the targeted site. (Being a Google tool, Skipfish should be particularly adept at comprehensively crawling a site, something many scanners struggle to get right.) When Skipfish discovers a new directory or POST parameter, it tests all possible <keyword> values and <keyword>.<extension> pairs from the selected dictionary to discover new files and directories. This is a great way to discover overlooked backup and development files, such as backup.tar.gz or database.csv, which shouldn’t exist or be accessible on the site.

The resulting site map is then annotated with the output from a number of active security checks for vulnerabilities such as SQL injection, shell command and XML/XPath injection, format string and integer overflow vulnerabilities. This site map approach to showing a scan’s findings is a useful way of displaying how a client connects to an application and all the possible resources they can access from within it, pointing to areas that may need further investigation.

Ratproxy is a passive assessment tool designed to transparently analyze legitimate, browser-driven interactions and automatically pinpoint, annotate and prioritize potential flaws or areas of concern. The proxy analyzes problems, such as cross-site script inclusion threats, insufficient cross-site request forgery defenses, caching issues, cross-site scripting candidates, potentially unsafe cross-domain code inclusion schemes and information leakage scenarios. This approach offers several significant advantages over more traditional active crawlers in terms of minimized risk of site disruption and good coverage of complex, client-driven application states in Web 2.0 sites.

Skipfish and Ratproxy focus mostly on testing server-side code. The DOM Snitch Chrome extension released in June is a passive in-the-browser reconnaissance tool to help developers identify insecure practices commonly found in client-side code. It intercepts JavaScript calls and tries to assess if the intercepted call can lead to cross-site scripting, mixed content, insecure modifications to the same-origin policy for DOM access, or other client-side issues. A great feature is developers can observe DOM modifications as they happen inside the browser without stepping through JavaScript code with a debugger or pausing the execution of their application.

While these open source testing tools for Web applications should be part of your toolkit, they’re not necessarily a replacement for the tools you already have. Some tools are more thorough, as these tools are designed to be fast and safely deployed against production systems without causing disruption. For example, Skipfish omits certain checks on purpose -- and others out of necessity -- to meet the challenge of remaining fast and simple. This means it doesn't satisfy many of the requirements outlined in the Web Application Security Consortium's Web Application Security Scanner Evaluation Criteria. For example, it doesn't check applications against a database of known vulnerabilities.

All three tools are relatively straightforward and easy to use, so even less experienced developers can use them to test their code, and of course they’re free. All three tools should be used during the verification phase of your Web application security testing program, as they use different methodologies than most other tools and support a variety of Web frameworks and mixed technology sites.

This was last published in October 2011

Dig Deeper on Web application and API security best practices