Splunk allows a variety of inputs, including logs, configuration files, traps and alerts, device and system messages, scripts, and performance data from applications, servers and networked devices. The software monitors file systems for configuration changes, watches files and logs, and can connect to network ports to receive syslog, SNMP and other network-based data. The Web interface uses drop-down boxes, making it easy to select a file to monitor, such as an actively growing log file, showing the most recent updates first.
Point Splunk to a file and it will intelligently parse the file, working out the event-types and normalizing a multitude of timestamp formats across different log types. Data is parsed and indexed on-the-fly, while raw events are kept for review. The data is secured with an MD5 hash using a PKI signature to detect tampering and point out gaps in the log where specific data may have been deleted. The tool keeps an audit record of who administers the system and who accesses data.
Splunk supports free-form search, but has a few tricks up its sleeve, such as a pop-up window with common events and values from logs to allow the administrator to zero in on specific behaviors, such as server errors.
Searches can be run on a schedule and set to trigger notifications or actions based on search results. Alerts can be used to monitor user or system activity and can trigger based on event types, event source or even the number of events; alerts can also trigger scripts to perform an action, such as restarting an application or service when it detects a condition. Notifications can be sent via email, RSS or Simple Network Management Protocol (SNMP) to other management consoles.
The biggest difference between the free version of Splunk and the commercial version is a cap that limits the maximum indexing volume to 500 MB per day. While meant to entice users to try Splunk, this volume cap may suffice for many small organizations.
Splunk works across a variety of platforms and deploys fairly quickly. It has an intelligently thought-out interface, which makes it relatively easy to use. If your company is looking for a product that offers log collection, review, searching, parsing and alerting -- for free -- Splunk may provide a smart and secure way to get the job done.
About the author:
Scott Sidel is an ISSO with Lockheed Martin. For more recommendations from the author, check out Scott Sidel's Downloads.
This was first published in November 2008