Problem solve Get help with specific problems with your technologies, process and projects.

Analyzing MSSP providers' log files for IT security events

Analyzing firewall, Windows server and antivirus log files can seem like an endless and tedious task, especially for an understaffed security team, but it's extremely important for detecting IT security events. Read more in this network security expert response from Dave Meier.

Recently, a reader asked our network security expert, David Meier, the following question:

We utilize an MSSP to collect various log files and issue alerts to us for specified conditions. We are looking into our firewall, Windows servers and antivirus logs to determine which events we want to be alerted about. However, there are tons of different types of events in each log type that could be of interest. What would you say are the most important instances to be alerted about?

First things first: You're off to a good start by being proactive in collecting log files from key devices in your network. The list you've provided is a good place to start, and it's adequate since the number of logging messages you're being presented with is overwhelming. So let's break this down based on the device types you've listed.

First up is the firewall. From the sounds of it, you may just have one for your organization, and it's more than likely sitting on the edge of your network between your internal networks and ISP. If that's the case, the device is probably creating the bulk of logs you're seeing because, quite simply, it's exposed to the public Internet. Most enterprise firewalls will let you configure the level of detail that they will transmit to the logging facility. Generally, I would first work through an aggregate set of data for a period of time -- starting with a smaller subset -- something like a day's worth. At that point I would skim through the data, sorted by identification number. Most systems will let you configure pruning to get rid of unwanted messages. If your firewall logs are turned up to a level such as "Informational" or "Debug" (the highest syslog levels), you're going to have a lot of data to work though, since almost every event that happens on the firewall will be logged in those scenarios.

Most organizations tend to believe that all blocked logs are the ones of most importance. ... [However,] some of the more interesting logs are from 'Allow' events to critical assets.

Most organizations tend to believe that all blocked logs are the ones of most importance. While I'm not discounting the fact that those are useful, some of the more interesting logs are from "Allow" events to critical assets. You probably have rules in your firewall allowing specific traffic in, maybe to your DMZ, or from a B2B connection. Generally those are the holes that will be most commonly used as attack vectors, especially if they're configured in a manner that leaves them wide open, i.e. blanket subnets. While those types of events may not show up as a "Deny" in the logs, they could be critical in noting that a box is being accessed from an unusual source.

Unfortunately, most organizations don't do egress filtering. Inside clients shouldn't be allowed to go just anywhere without some level of proxy or filtering services. Your firewall logs can be a great source of information on what's actually leaving, or trying to leave, your network. Since all of this may seem overwhelming, break the task into chunks that focus on critical assets from the ingress logs, and slowly work your way through key assets of your environment.

When I think of Windows server logging, a myriad of potential logging options comes to mind. Getting an idea of what events are a part of normal operations in your environment is key to discovering events that need attention. This can be a simple undertaking if your Windows servers are only using directory services for authentication. Conversely, things can quickly become complex when considering all of the other services that may be pertinent.

For more information

Read more about implementing firewall egress filtering.

Check out this screencast on Smoothwall, an open source firewall.

Learn how to estimate log generation rates with this expert advice.

If baseline analysis isn't something your organization has time for, there are options that can help you along your way. A tool that comes to mind to help make the process easier is OSSEC HIDS. This is free, open source intrusion detection software that includes a correlation and analysis engine and can help you pull together a relevant baseline. The best part is that, since you're using an MSSP, you can easily build a system where OSSEC can feed pertinent event information up to your provider, which will help you gain visibility through the analysis being done by the software. This can also tie in to your antivirus logs, as OSSEC can natively read those logs and help by alerting you during an event situation or weed out a deviation from baseline operations. There are other solutions that are similar, but OSSEC is an option that is fully free and open source, while also having a paid-for support option should your organization require one.

If there's one takeaway from this, it's that cookie-cutter answers to security questions generally aren't the best. Though there are commonly used software and hardware technologies in many companies, your environment and assets are unique, and critical data will be different as well. Keep that in mind when crafting policies and building supporting systems that are right for you.

About the author:
David Meier is a security consultant specializing in network architecture and current (and realistic) threats. He has designed and implemented solutions for the Air Force and Fortune 100 companies. David is also a contributor at security research and analysis firm Securosis.


This was last published in May 2010

Dig Deeper on Secure SaaS: Cloud application security

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.