Auditors are prodding companies to think about centralized log management in order to ensure control over scattered data, said Trent Henry, senior analyst at Burton Group: "So we have one place that can keep the information and have proper IT controls over the data to make sure it's not tampered with or lost or accessed by people who shouldn't, and that those policies are enforced."
No one compliance requirement is driving interest in log management, Henry said. A couple years ago, SOX was the top concern since it spurred most new audit efforts but now log data is important for demonstrating an organization's controls for a variety of regulations, he added.
But Dave Shackleford, vice president at the nonprofit Center for Internet Security and a SANS instructor, said the PCI Data Security Standard in particular is helping to make log management a hot topic in the enterprise.
Companies are figuring out that "they already have a lot of the information that they need to get a good bit of the way towards [PCI] compliance, they just don't have the tools to take that information and do anything with it," he said.
Log management tools can help organizations drill down and look for specific data strings such as full track data from credit cards; PCI prohibits storage of such information, so companies can then take corrective action.
The log management market includes tools from LogLogic, LogRhythm, Splunk, syslog-focused products such as Kiwi Enterprises' Syslog Daemon and freeware like Unix's syslog daemon. Also, security information management (SIM) vendors have begun tailoring their product lines to meet the demand for log management by offering options that focus on providing more storage capacity than correlation capability.
"Whether people want to acknowledge it or not, we're generating a significant amount of log data in any enterprise environment and there's a lot of cost associated with generation, collection and storage of log data," Leek said.
Without any control over what's being logged, companies can spend a great deal of time and effort searching through log data during an incident investigation or when trying to troubleshoot an IT problem, he said. Inconsistent logging formats and relying on homegrown scripts for analyzing and managing logs contribute to the difficulty.
Not having control over what's logged, stored and who has access to it can also create problems for a company that does business internationally because retention and privacy laws differ from one country to another, Leek said. For example, in France, log data containing personally identifiable information can be retained for a maximum six months while Russia requires some log data be kept for five years.
Deploying a log management system can streamline compliance and reduce the amount of resources needed to respond to numerous IT, security and audit requests for log data, Leek said. It provides the segregation of duties needed for various compliance purposes and also can guarantee chain of custody for forensics investigations. In addition to manpower savings, a centralized system reduces hardware and support costs.
Solid, enterprise-class tools for log management have come into the market in the past couple of years, he said. In particular, some tools provide for centralized management without storing log data in one place, which allows companies to comply with individual country laws.
Shackleford said a company looking to buy a log management solution should first consider their current volume of log data: "That could make or break a technology decision because some of the players don't have support for big-time storage."
Another consideration is the platform diversity in their environment; homegrown and legacy applications may not fit into standard logging formats, he said. While log management vendors say they parse any data, some make it easier than others.
Other factors to weigh when making a purchase are scalability and a vendor's viability, Shackleford added.