Information Security

Defending the digital infrastructure


Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Prospective Buyers Want Answers

SANS WhatWorks The SANS Institute's WhatWorks program identifies three critical areas of concern for security managers.

The SANS Institute's WhatWorks program identifies three critical areas of concern for security managers.

Nearly 500 software and hardware vendors offer more than 1,700 security products and services. Some are spectacularly valuable; others are no more useful than an electric fork. It's difficult to keep track of them all, and even more difficult to learn which will make an important improvement in the defensive posture of your organization.

Prospective buyers want answers to questions such as:

  • How well will the product reduce specific threats?
  • How difficult is it to implement?
  • How well does it work with other security and system management products?
  • What can go wrong during an implementation?
  • How good is technical support?
The only people who have good answers are those who actually use the products, but prospective users don't usually have access to those people. So to help each other and the community, more than 100 user organizations have begun sharing the lessons they learn in deploying dozens of security products.

In interviews conducted by SANS and supplemented with questions from dozens of other users, they describe the actual benefits received, the problems they encountered, how they justified the products, what it took to implement them, what went wrong, how technical support worked, and much more.

To determine which products should be subjects of WhatWorks interviews, many of SANS' 66,000 alumni participate in a continuous survey of product-buying patterns that identifies areas in which they plan to invest (see chart below), and product areas where they've already purchased and plan to make additional substantial investments in the next 12 months (see "Additional Investments").

New investments in laptop encryption, log management and application security testing tools top the agenda of SANS alumni who see the need to address data protection, regulatory compliance and emerging threats to applications.

Many more user stories are shared at the periodic user-to-user information exchanges called the WhatWorks Summits. In many cases, what users tell each other is significantly different from the stories they first heard from the vendors. The three greatest values of these exchanges are that they help buyers avoid over-promising to management the value of new security tools; help buyers learn insider tricks for getting full value and avoiding major problems with security products; and help buyers identify short lists of products that actually work.

The following are lessons learned during 2006 in the security product categories in which users are making the largest new investments--log management and security information management (SIM); laptop and mobile data encryption; and application security testing and secure coding training.

First-generation security information management systems rarely lived up to the ex-pectations raised by their proponents. Mixing data from various security products provided a lot of information, but little that could be acted upon. The investment in using these early tools was justified only in rare situations where extraordinary technical professionals amplified their value. During 2006, however, increasing regulatory demands caused buying interest in log management to skyrocket. SIM vendors joined pure-play log management vendors in responding to the frenzy of regulation-driven interest.

Gears in Motion
The SANS Institute is the largest cybersecurity school with more than 66,000 alumni. SANS also operates the Internet's early warning system, called Internet Storm Center (, and publishes original research on hundreds of topics in information security and information assurance. Listen to the SANS WhatWorks interviews at
Last July at SANS' Log Management Summit, 27 organizations with experience in log management and SIM shared the lessons they learned with 180 organizations about to acquire log management solutions. Among the highlights:
  • Organizations that use log management solutions only to produce regulatory reports are wasting a major opportunity for improving security, and also wasting a surprising opportunity to improve the relationship between security and operations managers.

  • The highest payoff from log management products appears to come from the daily "hot issues" reports that identify events that should not be happening--especially useful for malware and spyware identification--which then lead to action, and from the capability to perform forensics to determine what actually happened when a security event has been discovered.

  • When users cannot gain access to the network or to an application, operations and security people often find themselves at odds over who is at fault. A comprehensive logging solution provides access to data that can often isolate the problem and identify the solution quickly--eliminating conflict between security and operations staff.

  • Log management can help deter insider fraud or destructive activity by system administrators because it is much harder to cover up malicious activity when logging is enabled and when admins have no access to rewrite those logs.

  • The greatest problem is getting far-flung units in an organization to generate and share logs with the centralized log-management facility. Best solution: give each organization supplying data active access to the log management data and daily reports.

  • Another common problem is capacity on log management and SIM appliances. Most small and medium appliances are overrun with data in short order. Only the largest and most expensive appliances provide satisfactory capacity.

  • Log management also helps identify systems infected by viruses and other malware. When those systems try to infect others, firewall log entries are created. Close monitoring of those logs identifies unexpected traffic patterns. Drilling down identifies the infected system or systems.

  • Log management can enhance law enforcement efforts. Fresno County (California) used its logs to provide sufficient data to gain the arrests of two embezzlers, one drug dealer and one person who had fixed tickets. It also led to 15 terminations for computer use that did not conform to policy.

  • Other interesting applications: Finding inappropriate Web surfing and other inappropriate computer use, and finding evidence that a bank was the target of a phishing attack.

CIOs and their staffs at more than 6,000 organizations are implementing or evaluating solutions to encrypt the data on their laptops. They are doing this because their chief executive officers have told them to make sure they are protected from having sensitive data lost or stolen. They are not just trying to protect the information--the CEOs' own reputations are on the line. They want laptops encrypted, now.

At last September's SANS Laptop Encryption Summit, 18 organizations with experience implementing enterprise-wide laptop encryption shared the lessons they learned with 220 organizations that were evaluating encryption tools and planning for laptop encryption deployment. Among the highlights:

  • Organizations that acquired enterprise laptop encryption discovered that at least two vendors provided misleading responses to the request for proposals. The vendors provide "bottom line prices that do not include all the required elements," users said.

  • Organizations have found that using versions of several common third-party utilities and functions in Windows will make the encrypted data unreadable and unrecoverable. Exam-ples include Symantec Ghost and Windows Safe Boot.

  • Data loss also occurs because users who implement encryption try to take shortcuts. Some do not back up their data, and others skip the disk cleaning that most encryption vendors strongly recommend prior to encrypting a disk.

  • Most organizations choose full-disk encryption rather than file encryption. With user-controlled file encryption, the organization has no confidence that all sensitive information was encrypted on a lost or stolen laptop. That uncertainty exposes the organization to liability to disclose the loss of data under state breach disclosure laws.

  • Onboard hardware encryption is already being delivered by disk drive manufacturers and will be available from laptop vendors midyear. Hardware encryption users report that the technology removes most of the pain involved in deployment and management of laptop encryption, albeit at a higher price than software encryption.

  • Windows encryption functions built into Vista provide most of the benefits provided by third-party encryption tools. However, one critical function--enterprise management of the encryption process--is not yet available from Microsoft, so the third-party solutions continue to be worth the investment.

  • Among the most important features in laptop encryption are the need for the process to be automatic and safely reversible for users, and the need for bulletproof key recovery.

Where are security pros planning to add substantial investments in the next year? Standby technologies like firewalls and IDS still command resources, while the need for skills development emerges as a priority.

In July 2005, the SANS Institute revealed new data showing sophisticated attackers had altered their techniques and begun targeting application vulnerabilities more than system vulnerabilities. This announcement was a wake-up call for CIOs and security officers. Most of their efforts had gone into securing Windows, UNIX and their services. Now the CIOs had to worry about attackers using their applications against them.

Savvy users employ four types of defenses to protect their applications:

  • They intensely train users on secure coding techniques, and test them to be sure they learned to recognize the common mistakes and how to avoid them.

  • They test their programs using source code analysis tools.

  • They test their Web applications using Web application security scanners and penetration testing.

  • They deploy Web application firewalls and application-aware intrusion prevention systems.
Each of these defenses has matured greatly, especially in the past six months, so it may be time to update your application strategy.

Training and Testing
Until this year, training in secure programming was largely ineffective, because programmers had no way of measuring their mastery. Most teachers offered general rules but rarely reinforced the training with hands-on exercises focused on finding and fixing errors in code. The programmers listened, thought they understood, then went back to work and made many of the same mistakes they had made before the training.

In the past six months, more than 120 organizations have established a common definition of the rules and best practices in secure programming (in four sets of languages: C/C++, Perl/PHP, Java/J2EE, .NET/ASP) and helped to build a Secure Programming Assessment examination that allows programmers and their employers (and customers) know whether they have mastered the critical elements of secure coding. Information on the exam is available at

Just the promise of the exam has already begun to affect the content and effectiveness of secure programming training. Instructors have begun adding hands-on exercises and their students are paying a lot more attention because they know they will be tested. Even more importantly, the exam is being used by faculty at universities and community colleges to justify integrating secure coding training into the core curriculum. They don't want to be embarrassed when employers hire their graduates and learn that the programmers don't know how to write secure code.

At the same time, organizations that buy software intend to use the exam to find out whether the consultants and software companies have programmers who know the common security weaknesses in programs and how to find and fix them.

Source Code Analyzers
Early source code analyzers overwhelmed users with so many false positives that they lost their value. In recent months, however, they have gotten a lot more intelligent.

Although the false positive rate is still high, it is no longer overwhelming. Source code analyzers, like those from Fortify and Ounce Labs, are becoming trusted tools of application development teams and being embedded into application development tools. Source code scanners have their greatest value when developers use them before delivering code.

Smart software buyers write into their procurement documents that the developers are required to run multiple source code analyzers and deliver the results prior to delivery. Developers generally abhor demonstrating incompetence, so they usually fix the flaws before delivering the test results.

Web Application Vulnerability Scanners
The most common errors in Web applications--cross-site scripting and SQL injection--are very hard for source code analyzers to reliably identify. Web application vulnerability scanners, like the ones from SPI Dynamics and Watchfire (AppScan), have offered a powerful defense from their first deployment. They simulate the activities of attackers using a vast array of tests. Although not perfect, they provide great confidence to senior executives asking whether a new Web application is ready to be deployed.

Like code analyzers, Web vulnerability scanning should be required prior to accepting software from any vendor. Make the vendor test and deliver the reports before delivery is taken.

Web Application Firewalls
An alternative approach to protecting Web apps is to scan the data coming to those apps for things like character strings often used in SQL injection or cross-site scripting attacks. Web application firewalls have a huge job in trying to keep up with new attacks in different languages. They are an important part of defense in depth, but should not be relied upon alone.

Article 5 of 21

Dig Deeper on BYOD and mobile device security best practices

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Information Security

Access to all of our back issues View All