Review system event logs with Splunk

Logging and log review are two of the most difficult challenges faced by security pros. Even if a person is dedicated to nothing but log review, he or she would be quickly overwhelmed by the volume of information and the tedium of the task. That's why dedicated log-review systems are a must, and there's a tool for the job that's simple but powerful -- and free.

    Requires Free Membership to View

Splunk is a great tool for grabbing all sorts of network data, making it simple to search, providing triggers and alerts, keeping the data secure with granular access controls and offering controls for data audit and data integrity.

Splunk allows a variety of inputs, including logs, configuration files, traps and alerts, device and system messages, scripts, and performance data from applications, servers and networked devices. The software monitors file systems for configuration changes, watches files and logs, and can connect to network ports to receive syslog, SNMP and other network-based data. The Web interface uses drop-down boxes, making it easy to select a file to monitor, such as an actively growing log file, showing the most recent updates first.

Point Splunk to a file and it will intelligently parse the file, working out the event-types and normalizing a multitude of timestamp formats across different log types. Data is parsed and indexed on-the-fly, while raw events are kept for review. The data is secured with an MD5 hash using a PKI signature to detect tampering and point out gaps in the log where specific data may have been deleted. The tool keeps an audit record of who administers the system and who accesses data.

Splunk supports free-form search, but has a few tricks up its sleeve, such as a pop-up window with common events and values from logs to allow the administrator to zero in on specific behaviors, such as server errors.

Searches can be run on a schedule and set to trigger notifications or actions based on search results. Alerts can be used to monitor user or system activity and can trigger based on event types, event source or even the number of events; alerts can also trigger scripts to perform an action, such as restarting an application or service when it detects a condition. Notifications can be sent via email, RSS or Simple Network Management Protocol (SNMP) to other management consoles.

For more information
Learn about the latest efforts to develop a common logging and audit standard.

Interested in how to mine enterprise SIM logs for relevant security event data? Read more.
For security, Splunk uses SSL over TCP to secure the data-path. User sessions with the browser are performed using HTTPS over SSL. Roll-based access supports multiple types of users and comes with pre-built roles that can be modified or supplemented. Splunk integrates with Active Directory, LDAP and other directory services.

The biggest difference between the free version of Splunk and the commercial version is a cap that limits the maximum indexing volume to 500 MB per day. While meant to entice users to try Splunk, this volume cap may suffice for many small organizations.

Splunk works across a variety of platforms and deploys fairly quickly. It has an intelligently thought-out interface, which makes it relatively easy to use. If your company is looking for a product that offers log collection, review, searching, parsing and alerting -- for free -- Splunk may provide a smart and secure way to get the job done.

About the author:
Scott Sidel is an ISSO with Lockheed Martin. For more recommendations from the author, check out Scott Sidel's Downloads.

This was first published in November 2008

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.