Information Security

Defending the digital infrastructure


Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Log management reins in security and network device data

Learn how to manage log data from security and network devices in order to manage security events in real time.

Enterprises are swimming in a sea of logs. The deluge includes logs from servers, security systems such as firewalls and IDSes, events from network infrastructure devices such as routers and access gateways, and from various software and hosted services. Making it even more overwhelming is that the information isn't necessarily collected in a way to resolve security incidents in real time, or to troubleshoot situations that involve multiple segments of the enterprise network infrastructure.

Increasingly, however, IT administrators are under pressure to get a handle on their logging practices and manage log data. Regulations such as SOX and HIPAA require some type of audit trail, making log management critical for demonstrating compliance, while the Payment Card Industry (PCI) Data Security Standard specifically calls out the need for log review. Also, the latest changes to the Federal Rules of Civil Procedure (FRCP) require better log collection for legal evidence.

"We have seen a shift in the market toward regulatory and government-based standards to drive purchases of log management systems," says Chris Pick, vice president of products and marketing for NetIQ.

Part of the challenge is the need to look at logging from an enterprise perspective, and move toward having a common and centralized repository for all logging data.

The ultimate goal is to have this single repository used for a variety of purposes, from satisfying auditors and responding to e-discovery requests (see "How to Deal with New E-Discovery Rules") in the event of a lawsuit, to managing real-time security threat analysis and network and applications troubleshooting.

"Logging standards and practices typically do not exist across an organization, and they are difficult to enforce even if they do exist," says Jay Leek, manager of corporate IT security services at Nokia.

Three Steps Toward Better Logging
Despite the complications, there are three simple steps organizations can take to make logs more manageable.

  1. Separate your logging needs into three functional areas and look at what you need from each. These areas are the log collection process, the data repository where these logs are stored and the business analytics surrounding their use. Often, log management and SIM tools will serve one or two of these functions or perform different kinds of analysis, driving many enterprises toward buying separate products.

  2. Consider the chain of legal custody when designing a log management scheme, so your log archive can be used as evidence and stand up in court. "There is a lack of controls over the process for accessing logs, and there are serious questions about who should be able to view sensitive data and how to handle the chain of custody when handling log data too," says Nokia manager of corporate IT security services Jay Leek.

    Generally, log management tools focus more on preserving a chain of custody than SIMs, which normalize data for correlation and analysis. "You want to have the shortest custody chains possible," says ArcSight's Hugh Njemanze. "We put our log management system in front of our SIM, so it becomes the repository of record. This means that the SIM isn't part of the chain of custody."

    Chris Pick of NetIQ adds: "You want any event collector to digitally sign the collection to ensure the nonrepudiation of that data source and to make sure your logging events aren't tampered with." For example, NetIQ provides agents that will guarantee the delivery of log information into its repository, and digitally signs this information too.

  3. Balance costs and benefits with security and compliance needs. "The cost of noncompliance can determine the overall log management requirements for the enterprise," says Pick. "It isn't a one-size–fits-all type of offering."

    Also, steer clear of creating homegrown solutions because the support costs can add up over time. "Homegrown log management scripts are expensive and exist in every organization. Generally, only one person knows how these scripts work, and the environment is often constantly changing," says Leek.

Leek suggests enlisting the aid of the internal legal staff to help bring about this unification; there are numerous regulations that require logs to be kept for varying time periods, and in some cases disposed of after established deadlines for privacy reasons. For example, HIPAA mandates a seven-year retention period, while PCI requires one year. It gets more complex for global corporations, where European and Asian laws come into play. And logs need to be intact if they are going to be used as evidence in civil or criminal proceedings, placing other requirements on their use. Given the contradictory legal requirements, having lawyers as partners becomes essential.

"We need to bring together IT and legal departments to help put in place overall enterprise IT logging standards," adds Leek. "Lawyers tend to not be very technical people. If you can make it simpler for them, they will make things simpler for you. But don't let your legal department run a logging project; instead, incorporate their advice, and try to speak the same language."

A primary objective should be to prevent departments from setting up their own log management tools, creating multiple places where logs live, says Matt Stevens, CTO of the information and event management group at RSA, the security division of EMC. "You need analysis across the enterprise and to make it accessible for all users, and log management needs to be an element of the overall management infrastructure," he says.

Log management is also now part of the overall network security infrastructure. As blended threats become more frequent and more corporate applications make deeper use of the Internet for connectivity, having a unified logging repository becomes another tool in the security chest to protect the enterprise.

"It is not your father's security landscape anymore," says Robert Whiteley, an analyst with Forrester Research. "Nowadays, threat mitigation is deeply embedded into the overall network infrastructure. But how well you maintain your environment is critical, and there is a huge range in terms of how data can be exposed for analysis and manipulated."

In some cases, a log management tool is seen as the first step in the analytical chain. But the market for such tools has become confusing, with dual product lines from log management vendors that have branched out into the security information management (SIM) space, and SIM vendors that offer other versions of their products for log management.

"Increasingly our midsized customers want to just get their auditors off their back and don't necessarily want to implement a full-blown SIM," says Tracy Hulver, vice president of marketing and product development of netForensics.

The issue is that the two approaches--SIMs and log management tools--serve different masters and are often two different products, with SIMs focused on correlation and real-time alerts, and log managers emphasizing long-term archiving and evidence preservation. "Log management is more useful for ad hoc, after-the-fact investigation, while SIM is more concerned about codifying the business rules and notifying the security team to respond to a problem," says Hugh Njemanze, CTO and executive vice president of engineering of ArcSight.

As a result, some of the SIM and log management product lines have been developed independently, even those sold by the same vendor. For example, netForensics and NetIQ don't offer a common repository for their two product lines, although the former is working toward this goal and hopes to have a unified repository by the end of this year. "In the past we had a single repository for our SIM and log management product lines but there were some issues," says NetIQ's Pick. "Now we have a SQL database on the real-time side for the SIM, and have flat files that are indexed for the log archive server."

Another difference between log management tools and SIMs is how they analyze data. Most log managers do "free text" searching, which is useful for finding particular records that can be used for legal evidence. SIMs tend to normalize and analyze network events and can correlate different conversations between computers or IP addresses, which is useful in resolving incidents or tracking down exploits.

Choosing between a SIM and a log management tool depends on your organization's log management goals. If compliance and auditing requirements are your pressing issues, then start with a traditional log management tool. If you're more worried about breaches, start with a SIM.

In the end, you may decide you need both.

Whichever type of product you choose, reporting capabilities are critical. NetForensics' Hulver says most customers want many different items in their reports, so his company provides some templates, but also makes it easy for clients to create their own.

Customized reports are crucial for compliance purposes, but some aspects of the reporting process are also needed for real-time threat analysis. "We put all the information online so the security professional can solve problems immediately," says Jim Pflaging, president and CEO of SenSage. "You can also set rules that look into the past, and can do threshold and violation alerts based on long-term trend analysis. That also has advantages for compliance."

How a product handles ordinary network traffic flow is also important. Some products, such as netForensics' nFX Log One and RSA's enVision, don't offer the ability to import this information, making it more difficult to correlate particular network events with security breaches. "It is on our 2008 product road map," says Hulver.

In addition, IT staffs need to work with logging application program interfaces that can import other kinds of logs--such as those from custom applications--into a central repository, and query mechanisms to search through this archive to get the right kinds of information out of it. "Many times there are custom ERP applications that make use of .NET or J2EE that require their logs to be aggregated, so it's a must to have an extensible API to do that," says Pick.

Another factor in a purchasing decision is how scalable and extensible a solution will be, and what happens to the existing repository when new logs are imported or added. Vendors have adopted their approaches to address this issue. "We can easily extend our data schemas without having to change the underlying database or the collection process. We have built a relational approach without all the overhead associated with the relational database," says Pflaging.

"We developed our own purpose-built, object-oriented database that allows us to scale to billions of daily events," says RSA's Stevens. "We can easily generate metadata when new data sources are added to it."

How to Deal With New E-Discovery Rules
Federal regulations require companies to take stock of the data they keep and how they manage it.

Many different technologies, including log management tools, can gather information to help satisfy the latest federal rules on electronic evidence discovery. And that's a good thing because in a recent survey of IT managers by Osterman Research, more than half said they'd rather have a cavity filled than respond to an e-discovery request.

Last December, the Federal Rules of Civil Procedure were updated to address electronically stored information in the pre-trial discovery process.

The new rules require that companies be able to identify relevant electronic evidence in a timely and complete way, with exceptions granted for data that's not reasonably accessible or not kept as a matter of routine operations.

So before you head to the dentist, consider what kinds of tools you currently have, what kinds of information are already being collected, and whether any of this is suitable for e-discovery purposes.

"There are now 38 states with some form of mandatory disclosure laws," says Robert Whiteley, an analyst with Forrester Research. "It is only a matter of time before everyone has to worry about compromised assets and the potential for legal action."

In any legal challenge, first take a look at the time frames involved, and whether you actually have the data you think you have. Some data may already have been removed from your archives because of how you set up various systems.

Osterman's survey found that 25 percent of the organizations polled purge their email manually or automatically after 90 days or less. This may not be suitable when laws such as SOX, HIPAA and various European laws require archives of multiple years worth of information. Look at how your email system can create longer-term archives for these purposes. Nearly one-third of organizations Osterman surveyed admitted that, even if they had to, they would not be able to produce an email that is a year old.

"Organizations have had to pay fines in certain cases when they had destroyed or missing records," says Larry Dietz, managing director at security consulting firm Tal Global.

Products such as LiveOffice Managed Messaging Services can help store and index Microsoft Exchange email, set predetermined retention periods, and automatically archive messages to offload the active email servers.

Second, understand what kind of data is actually required for evidence. Email usually comes to mind first, but other data can be requested in these types of legal actions, including instant message logs. Reuters this year started selling Messaging Compliance Manager, which allows customers to log all Reuters Messaging communications for up to seven years.

However, legal requests can go beyond messaging applications. "Security logs can be an issue in cases of intellectual property theft or financial malfeasance," says Dietz. "The SIM might contain the pointers for what evidence is needed to respond to the lawsuit, rather than have the actual evidence themselves."

Third, if you do actually have the data, what steps are you taking to safeguard logs for evidentiary purposes? If and when you do get sued, you will have to show that the log data your systems have collected has not been tampered with, and that some form of digital signature is still intact.

"There is no clear point-by-point situation that is the court-approved method of collecting log information," says Anton Chuvakin, director of product management at LogLogic. "Everything is subject to case law and judicial interpretation. But if logs are collected in the normal course of business, and if this process were relatively solid and protected, then they would be likely admitted as evidence."


Standby in the logging world is syslog, which provides a framework for collecting and storing log data but has well-known performance issues and can drop some data during periods of high network use. Some vendors also support a more recent version called syslog-ng (for next generation) that includes delivery using TCP instead of UDP.

"Syslog-ng tries to solve that problem with guaranteed delivery, but that can slow down the collection process," says ArcSight's Njemanze. The trade-off is having a high-performance collector that misses log events but keeps up with real-time traffic analysis for threat mitigation, versus having something more complete but lags behind in real-time collection.

"When you are capturing all this log data you shouldn't be forced to filter or normalize any of it, because that slows things down," says Stevens.

As a result, LogLogic offers two different log management product lines. One stores its logs in a SQL database, while the other uses raw files. "It is important to do both," says Anton Chuvakin, director of product management of LogLogic. "Some users of log data want the flexibility to do visualization and compliance reports, while others want to be able to do full text searches."

"In practice, most of our customers tend to go with traditional syslog because they want to see current messages, even if this means that they lose a few in the collection process. Whichever method you employ, make sure that the system you use to capture logs has the capacity to keep up with the message traffic," says Njemanze.

"Syslog is pretty bad and has all sorts of issues, but it's also really common, and there are millions of devices that write to its format," says Chuvakin. "Sometimes convenience can override security concerns."

Without a doubt, log management is a tough task to tackle, but the security and compliance benefits it can provide have become essential. And while the market of available tools that can help ease the process is rather convoluted, it may become clearer as vendors hone their products. Both log managers and SIMs will continue to converge as vendors add features to complement and extend their product lines. For the next few years, however, it is likely that IT and security managers will need both kinds of products to satisfy multiple needs.

Article 8 of 15

Dig Deeper on Network device security: Appliances, firewalls and switches

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Information Security

Access to all of our back issues View All