This Q&A is part of SearchSecurity.com's Integration of Networking and Security School lesson, Application and network log management program planning. For more learning resources, visit either the lesson page or the Integration of Networking and Security School main page.
Log aggregation seems so "turn of the century:" everyone is already doing it, so why should companies spend time and resources addressing it now? The answer is because they probably have a huge blind spot: Most applications don't log well.
In this interview, SecurityCurve's Diana Kelley discusses application event log management with James McGovern, enterprise architect and scecurity thought leader for a Fortune 500 enterprise headquartered in Connecticut. McGovern explains why he believes application logging is the "final frontier."
Diana Kelley: Jim, don't all IT professionals already have everything they need to aggregate logs and use them intelligently?
James McGovern: A fool with a tool is still a fool. Having a network security engineer review the logs of an application for security breaches increases the odds of failure. Why? Because there's an interesting, and fine, line between leveraging developers and attempting to think that it is a simple matter of comprehensive documentation for someone else to review.
Generally speaking, it is easier to teach a developer how to code securely than it is to help a network security engineer understand logs within a software development environment.
DK: Are you saying developers need standards?
JM: Yes. If you were to look at the plethora of software products from Oracle, HP, EMC, etc., do you think they all adhere to a common logging format? They don't! If companies that make their living off developing software don't get it right, can we expect a large enterprise to? We need an [independent] organization to lead a cross-industry consortium to work on the challenges in this space related to standards, semantics and interoperability.
DK: I definitely agree with you on that! Interoperability and normalization of log formats, including the syslog standards work going on within organizations like the IETF, will help with log management processes. First by providing aggregator solutions more effective ways to gather critical log data, and secondly by helping parsing engines to derive better information about current, future, and past activity on the network and applications. Let's go back to the challenges many companies face now and how to address them.
JM: So, you've got a log management appliance. How many enterprises have ever asked their software vendors about log management integration at procurement time, or even RFP? Can this gap be closed by a few reusable clauses in procurement agreements?
DK: That's a great point: log management requirements don't need to be re-invented for every new device or system component RFP; re-use of existing wording reduces RFP creation time and, when written properly, ensures the requirements are defined consistently. What do you think is the most important use of log management for enterprises?
JM: It's the way in which we understand what is happening in the enterprise. If we can't understand activity and access, there's no way we can understand impact to the business. It sounds simple, but it's very complicated. The logs need to address access and activity or there's no way log management can parse it usefully. It is cause and effect.
DK: How else can companies improve?
JM: Don't treat log management like a reporting activity. We simply have too many lists and too many spreadsheets. More importantly, we need to do alerting not just on the characteristics of individual rows, but also on sets. For example, if you happen to have been smart enough to develop a log management "profile" for your application, you may know how many failed authentication events occur on average within a specified time period. Wouldn't it be beneficial to know if the number were higher than average or even lower to detect activity trends?
DK: What about outages? Do log management systems become blind sides?
JM: Logs are wonderful for outages. Having sat through many crisis calls with business customers and a major revenue-generating application being down, I can certainly say that the chaos of everyone gathering information in silos is suboptimal. Unlike current processes that involve speculation and heroics, logs are an actual fact of what happened. If you are going to assemble a team, it would be great to write reports in real-time to put together the pieces.
DK: Any final words of advice for your peers?
JM: Log management is not just about security. Imagine having the ability to aggregate logs from firewalls to applications such that you can analyze the customer experience for your top ten business relationships. Surely, current Web reporting and other coarse-grained approaches aren't sufficient to truly understand the customer experience. Well-designed application logging would take the customer experience into account.
About the author:
Diana Kelley is a partner with Amherst, N.H.-based consulting firm SecurityCurve. She formerly served as vice president and service director with research firm Burton Group. She has extensive experience creating secure network architectures and business solutions for large corporations and delivering strategic, competitive knowledge to security software vendors.
Dig deeper on Security Event Management