According to the 8th annual SANS Log Management Survey (pdf), not only are many organizations struggling to distinguish between normal and suspicious traffic, but also more than one-fifth of respondents
Logs provide the evidence of what's occurred on a system at any given time, but logs are of little use unless they are collected and analyzed properly.
However, enterprise security teams are often understaffed and underfunded, meaning expensive network log management and analysis systems may not be an option. Even if they were, the manpower to run them simply isn't there. But there is hope. In this tip, we'll examine free tools and simple tactics that, when implemented, can make it much easier for organizations to take advantage of security logging to spot anomalies and potential attacks.
Security logging, while essential for any sizable organization, isn't easy. Not only does the effort become more complex when collecting logs from many important systems, but also the time and effort in man-hours needed to pour over data, develop analysis and conduct ongoing tuning can be enormous. Many organizations can't even afford the cost and effort to purchase a commercial system. Fortunately there are free tools they can use to assist with log management and correlation to make life easier for log management systems administrators.
If the company doesn't have the budget to purchase a robust log management system from a large vendor, there are two good free log management tools worth checking out: OSSEC and syslog-ng. Both allow for data to be logged from systems and both also offer log analysis and reporting. A search engine can help find many other free programs available, but these two are the most widely used.
Since many of these free log tools rely on syslog, it might be necessary to install an agent on Windows servers to convert the Windows events logs into a syslog format. Downloading and installing the Snare agent will help the log manager bypass any limitations that might exist in parsing various log types. One thing to remember, though, is that when a logging agent fails, logs will stop being sent. I've seen systems that were up for long periods of time with an agent that crashed or that didn't come back up after a reboot. So it's important to be able to quickly determine when logs aren't being received. Run a weekly report on the events sent by each host. If you see a host's log volume that's lower than its established baseline or isn't sending logs at all, you have an area to focus on.
Splunk, although not completely free, has a limited version that can be used to review logs that might not have reporting or dashboard capabilities. The free version of Splunk is able to review 500 MB of logs per day, which can be useful as a supplemental log management tool if an organization is missing certain options from its free log manager and doesn't want to go over the limit.
Another trend we're seeing is the emergence of cloud log management systems that allow logs to be sent offsite without the need to manage on-premises systems mentioned above. This can lower the TCO for companies that don't have the resources to manage a log management system in-house or that want to have their logs stored offsite. One service that does this for free is Sumo Logic: It allows for 500 MB with retention for seven days and the ability to have up to three users managing the events; plus users can run reports, create alerts and search logs. It may not be robust enough for some organizations, but it may be worth experimenting with in order to get a feel for log management in the cloud.
From the editors: More on log management
Check out SearchSecurity.com's Integration of Networking and Security School lesson on security event log analysis.
Log retention is also important. Even if you don't have a commercial log management system and are storing logs locally, that storage appliance or server needs to have redundancy built into it. You don't want a simple disk failure to erase your logs. Backing up logs and having them sent offsite, or better yet, replicated somewhere else on the network (securely) is important. The Verizon Data Breach Investigations Report (DBIR) states that when viewing logs after an incident, in the majority of cases, the data was there to be found beforehand, it just needed to be viewed. Without stability in the log storage process, that option to go back to the past might be lost.
Signs your company needs help
Finally, while many of us assume what we're performing is "good enough" when it comes to streamlining log management, below are a few telltale signs that an organization's security log management process is in trouble. If more than one of these apply to your organization and the issues can't be rectified, it's time to call for outside help.
- There is limited or no way of automating alerts of logs within the company.
- If there is no way to alert on particular log events, the company won't know when it's had an incident.
- Administrators don't understand what is being logged.
- Not knowing what logs are coming from the systems is an issue.
- Not knowing what level of auditing is enabled on the system is also a problem. This might lead to a false sense of security.
- Not being able to log custom applications. Many systems and applications have custom logs that need to be parsed and stored. Verify that this is possible and happening.
- Forgetting to log third-party, cloud, mobile and virtualization systems.
- Performance issues are observed.
- Slow database that doesn't allow for flexible reporting or searching.
- No drill-down capabilities when searching for logs; speed is an issue.
- Use of out-of-date equipment or single points of failure in hardware that would allow logs to be lost.
- Not calculating the proper events per second (EPS) and losing logs due to saturation.
- Correlating logs for search purposes is an issue.
- Without correlation, log managers will try to create a handful of alerts or reports at best. Without this functionality, they can't see deeper into logs to search for security incidents.
- There's no process in place for monitoring and analyzing logs.
- No process for adding new systems to the log manager.
- Unaware of what logs are missing or which systems are not sending logs.
- No audits of systems to verify that logs are being collected from all systems.
About the Author:
Matthew Pascucci is a senior information security engineer for a large retail company where he leads the threat and vulnerability management program. He's written for various information security publications, has spoken for many industry companies and is heavily involved with his local InfraGard chapter. You can follow him on Twitter at @matthewpascucci or check out his blog at www.frontlinesentinel.com.
This was first published in September 2012