If you know that a security violation originated from a specific machine within the last 60 days, how would you determine who has logged on to that machine? All of the necessary data is sitting in the registry and log files of a Windows system, but you still need to sift through it all. How about if your company experiences a malware outbreak and you need to determine if a certain computer has been infected? You'd likely have to perform a manual registry search. The nature of such tasks can be tedious and time-consuming, but by using Splunk's security features, IT pros can search through the necessary data more efficiently.
In this SearchSecurity.com screencast, Keith Barker, CISSP and trainer for CBT Nuggets, provides a Splunk video tutorial to exhibit how to use the Splunk tool for security. Keith analyzes the features available in the free version of Splunk, including a powerful set of predefined searches that can be tweaked to achieve specific results. There are also plenty of apps that provide predefined searches for specific platforms, including Windows, Linux and Cisco firewalls. All of these powerful features are presented in an easy-to-use web browser layout, so any user that can navigate a browser will instantly be familiar with Splunk's design. With this Splunk video tutorial, IT pros can learn how to sort through avalanches of data quickly and easily, eliminating some of their most monotonous tasks in the process.
Editor's note: To make the video larger, click the go to full screen button in the lower right-hand corner of the video window.
About CBT Nuggets:
CBT Nuggets is a computer-based technology company specializing in cutting edge online IT training. Founded in 1999 by current CEO Dan Charbonneau, CBT Nuggets provides quick, easy and affordable learning by renowned instructors for individuals, small teams and large organizations. CBT Nuggets also offers a wealth of free videos on a variety of IT topics on the CBT Nuggets' YouTube video channel.
About Keith Barker:
Keith Barker, CISSP, a trainer for CBT Nuggets, has more than 27 years of IT experience. He is a double CCIE and has been named a Cisco Designated VIP. Barker is also the author of numerous Cisco Press books and articles.
Read the unedited transcript of the Splunk video tutorial below:
Welcome to this searchsecurity.com screencast. I'm Keith Barker with CBT Nuggets, and today we're going to feature Splunk. It's an amazing tool that can help us sift through the avalanches of data so we can find our own required needle in a haystack. Let's begin.
Besides being incredibly fun to say, Splunk comes in a couple of flavors. There's the free personal edition. There's also a commercial version as well. But the bottom line is we're going to feed in all of this data into Splunk. Now, what kind of data does the Splunk software tool eat? Well, it'll eat virtually anything. If a machine generated the output, for example syslog files, Windows event logs, the Windows registry, we can feed all of this data into Splunk. Then, we can do incredibly fast searches where it can sort through all of it and show us the relevant data that we're specifically looking for.
To make it easier, there's a boatload of predefined searches that we can either use and/or tweak or customize to fit our exact needs. The easiest part is it's all based inside of a browser look and feel. So all these apps working together, it looks like a normal browser window. So an average user who's comfortable with the browser can quickly and easily use this search tool.
Why use the Splunk tool for security?
To really appreciate the power of this tool, let's take a look at a few examples of how we might use this together. Number one, let's say we've identified that there's been a rogue access point, and we're curious whether or not a certain computer has associated with that access point within the last 24 hours, or last 30 days, or last 6 months. We can get all that information by parsing all the data on the computer. Or another example, maybe we had a security violation and we need to find out who logged on at a specific machine within the past 30 days, or 60 days, or 90 days. All of that data is sitting in the files, the registries and the log files, on a Windows computer. We just need to be able to sift through it all to find it.
The last example is: perhaps there is some malware, and that malware -- if a computer was infected -- left a telltale sign. How do we search that information very, very quickly without a tedious, manual registry search? The answer is Splunk. Splunk is … the free edition on a personal computer, can sort through all of the data you input and very, very quickly show us exactly what we're looking for.
How to use the Splunk tool
This is the Welcome page for a new install after you've just installed Splunk and it's currently running on a computer. What I did, first of all, I went to Manager over here, and I said, "I want to go ahead and manage my licensing," and I used this option to change my license over to the free personal edition. Once that was done, I said, "You know what? I now need to get data into the system. I need to tell Splunk what I want to have analyzed." Is it my registry? Is it my system files? What is it? So I went to Data Inputs, and under Data Inputs it's going to list everything that it's currently collecting, and if you want to add new ones you can click it right here.
So regarding registry, for example, I've got three inputs for my current registry. If I want to edit that, I click on Registry Monitoring and you can delete it, disable it, or you can Create New. So if you want to look at the entire registry or part of the registry on a system, you can do it right here. Then, use the convenient Back buttons to go back, just like a normal browser would. So we can drill down on virtually everything in Splunk. On every page, it lets you drill down.
So if we want to go back to the basics for the search, we can go to App and go to Search. Now, if you look at my events indexed, I've got over 2 million, and I dumped in a big portion of my registry. I dumped in some system files that I have on this computer, so I want them all to be analyzed. I want to show you how quick and easy it is to search for data. For example, let's say we're searching for a rogue access point and we want to know, "Has this computer ever been associated with that access point? Is it in the network list?" Well, that information is contained in the registry, tucked away in one of those 2 million data points, so we can search for it.
So to search for something, we just type in "search". For example, I know the access point started with 678. As we type, check this out. As we type in, it's going to show us how we're doing. It's like a little gauge of our progress. It says, "Okay. You've got 678. You've got 1,279 matches. For 6780, it's 57 and so forth. So if I want to narrow down the search even more, I can say, "Well, it was 6783-an, which it was. It's telling me in measurable terms, right here, it says, "You've got 10 of those." There's 10 matches. Now, where exactly are those matches? Well, all we need to do is click on Search right here. It will go out and find them for us, and tell us exactly where they are.
Now, this is for all time. This is all the data it has, no time limits. Or we can say, "I want to see within the last 30 days," and it's going to show us six of those events in the last 30 days, and then it shows us specifically where they're at. Now, the drill down part here is amazing. So right now, I'm looking for 6783-an. But if I want to source just from my Windows registry, I can click here and check this out. It changes my search to add it from just my registry. So I can dynamically modify my searches as we go.
If I scroll down here, we can see every one of the events, and there are six events, wow, in that one day, November 27, 2012. That's the only time it happened in the last 30 days based on the output of what it scoured my system to find. So that's one quick example of how we can use Splunk to search through mounds and mounds of data, over 2 million data points, to find the exact details that we're looking for.
Predefined reporting features
Doing searches is great. However, what if we want a whole bunch of prebuilt searches based on the platform we're using? One of the other functions that we can use are apps that are built for Splunk. For example, I just clicked on Manager and under Manager we can go to Apps, and under Apps we can say, "I want to find more apps online." So for example, if we install the Splunk app for Windows, it's got a whole bunch of prebuilt data inputs, searches, reports, alerts, and dashboards that I don't have to manually create.
Check this out. They have some for Linux, some Splunk for Cisco firewalls. They've got one for the Cisco security suite. If we go to the next page over, it gets even better. They've got a lot of great apps for some very common features, and they also have one down here for monitoring, Splunk Monitoring, which links into nmaps scripts. So the convenient thing is we don't have to manually create all of our dashboards and all of our custom reports. We can do so with apps to make it more convenient.
An example of the Windows app is right here. Because I installed it, if I click on Windows, it's going to give me the predefined information that is typically wanted or desired by a Windows device. So I have some hyperlinks here. But if I want to take a look at my performance, for example, on this system, I simply click on Performance. It gathers the data, and it puts it in. So any guesses what was happening right here? Well, during that time period, this computer was off and for that reason it has no CPU memory or disk reporting for it.
If we scroll down a little bit, it also has top processes for the last 24 hours. So these are just examples of some common reports that are prebuilt. So all we have to do is click and it will dig out that information and show it to us. One of the things I really like is that you can just go back. Use your Back browser to go back to the previous page. So there's the Performance dashboard. Here's a System Management dashboard.
Examining log files
Now, what's interesting to me as I look at this report right down here is the longest running logins. I've got an anonymous login for 185 days. That seems interesting to me. So if we wanted to we could go ahead and run our own custom searches, and let's do that right now. I'm going to go back to the app for just general searches. So let's do a search for the keyword "authentication". This is going to search out of all the data we've put into the hopper, the registry, any log files, and system files that we've told it to pay attention to, and I'm going to go ahead and press Enter. It's going to search, and we can say the last 30 days, we can say the last 4 hours, and it gave us those reports based on those timestamps.
So I haven't had any authentication messages, anything with "authentication" in the last four hours. I must have been logged in longer than that. Let's say the last 30 days. It's going to grab all of that data and bring it up. I've got 938 events regarding authentication. Now, the great thing about this is that virtually everything I can drill down on. I can drill down on a specific date. It will show me that down here. I can click on WinRegistry here. It'll add that to my search criteria for only that keyword within WinRegistry.
I also have on the left-hand some what's called "interesting fields". If we take a look at those, let's do a quick search based on account name. Now, to do that we can go ahead and say, "You know what? There are seven different elements for account name." We can click on this little graph option, and it's going to show us that we've had logins from System, Keith, Network Service, etc. We could generate a new graphic just on that. We can say, "I want to do the top values based on time." Click on that, and it will give us a new graphic just based on that.
So now I've got this anonymous login. Now, that should scare me and it does. So we want to say, "Now, what the heck is that?" We could hover on Anonymous Login, click on it, and it would take us to the exact data where that came from. So in this case, it's saying that this came from my security logs. If I want to see all the details, I can click on Show All 60 Lines, and then scroll through it to identify exactly what's going on with that. So that's a great way, again, to drill down and find out what is really going on.
If we wanted to go back again, [it's just a quick] Back button to take us to that previous search. So if we wanted to go back to the original search, we can just take off the filter based on by account name, and that should bring us back to the original search. I'll go ahead and click on Search to bring us back there. We did the previous filter based on the account name. Let's go ahead and go down to Security ID, just as another example.
So I've got six there, and if I just click on it, it'll show me the details for it. I can click on any one of these, it would drill down to that. But I could bring up another graph for that. Each one of these, we could create a custom dashboard out of it, and then have that as a preset, so we could go and look at that on-demand whenever we needed to see it. We've had a couple of measurable examples of how we could use Splunk to sort through mounds of data to find the details.
First, we looked for an access point that we had associated with. Secondly, we looked at authentication information. What if we're looking for some aspect of the registry? For example, maybe we're looking for a needle in a haystack. We can go back to Search with the nice clean page, and let's search for "haystack". That's telling me right there as I start typing it in, it's saying, "You know what? You've got two that match 'haystack'." If I wanted to match "needle" as well, I could type in "needle". What "needle" is saying, it's saying, "You've got two 'needles' as well."
So if I wanted to do a search... This is a search for "needle" and "haystack". If I wanted it "needle OR haystack", I would do like this with a capital O-R. That would search for "needle OR haystack". It's going to search all the data and show me exactly where that is. So in this case, I do happen to have a needle in a haystack. So in a situation where we have a malicious application that left a telltale sign in the registry somewhere or some keyword, we could search for that in any of the input that we fed into Splunk, whether that's a log file, a system file, the registry, or any of the other data that we have fed into Splunk.
Having mountains of information is one thing, and finding critical data and security-related data inside of that mountain is quite another. The free tool called Splunk, the personal edition, is an amazing tool to assist us in finding and sifting through all of that data. The browser-based interface makes it super-easy, and the predefined searches and downloadable apps make it that much more powerful. It can assist anyone in finding that needle in their data haystack.
My name is Keith Barker with CBT Nuggets. Hey, thanks for watching this SearchSecurity screencast. For more screencasts visit searchsecurity.com/screencasts.