This article describes how to use basic default Unix tools to investigate a potential security problem. Basic forensics is a very important part of the System Administrator's duties and should not be over looked. It is often important to understand that the tools needed to perform good forensics work are already at your fingertips, and there is no reason to spend a lot of money on software. The example below is based on a web server...
conf file but could be a disgruntled employee, hacker, embezzlement, and the list goes on.
It's 2:00 AM and you have just been called in to fix a Web server. It turns out that somebody modified the httpd.conf file at 7:20am. When the Web server restarted during a scheduled reboot at 1:00 AM the new changes from the conf file took effect. So, you begin the first steps of good forensics while thinking about how to thank the person that made you come in at 2:00 am.
Here comes our friend the command last. We know that the file was modified at 7:20 am by using the ls command to get the last time of change.
If your system has been running for any time at all, there may be thousands of entries from the output from the command last. Fortunately, we can pipe last through grep and make the output much more manageable.
The command below should render the information we are looking for. It is important to place the proper amount of spaces in the command to capture the required output. For the purpose of this article we'll use Jan 7 as the date that the incident occurred.
last | grep "Jan 7 " root pts/13 serverA Tue Jan 7 06:39 - 06:39 (00:00) user2 pts/13 serverA Tue Jan 7 06:39 - 06:39 (00:00) user2 pts/13 serverB Tue Jan 7 06:39 - 06:42 (00:03) user2 pts/13 serverA Tue Jan 7 06:39 - 06:40 (00:01) user1 pts/13 serverA Tue Jan 7 06:38 - 07:38 (01:00) user2 pts/13 serverB Tue Jan 7 06:38 - 06:38 (00:00)
In the example above we can see that user1 was on the system from 6:38am to 7:38am. The user came from ServerA and was on the system for 1 hour. A safe assumption would be that user1 has something to do with the edit of the conf file. The output shows that user1 was the only user logged on at the time of the edit. The possible number of culprits can be further narrowed down by looking at who has permissions to change this file. This file access for this conf file should be rather small since you must have tightened up access to that file (you did tighten access up to this file right?). Look at the file permissions using the ls command and see if user1 would have permissions to modify that file.
Some users never log out or even worst generic accounts. If user1 is a generic account you will need to go to serverA and run last again and keep on tracking down until you get a workstation. The workstation may be your last step. If it's a minor issue you can talk to the user. If it's a major infraction, such as embezzlement, disgruntled employee or other criminal behavior you will want to take an extra step. Look at badge access, security cameras and any other security features of your company. Last thing you want to do is accuse someone without being 1024% sure.
If the permissions could not allow the user to modify the file, then you will need to continue looking. It may be a user that logged in a week ago and has not signed out.
Last will show you users that are still logged in.
Last | grep "still logged in"
root ttyv2 10.0.132.28 Fri Jan 3 08:12 still logged in
Wait a second. Isn't that your workstations ip? Yeah, don't you remember you made a change for your Aunt Selma's Web server but instead of saving it in a temp file you accidentally made the changes on a production server.
It is important to remember when performing a forensics study do not assume that you are looking for someone acting out of malicious intent. Most issues that occur in a company are simply mistakes. To err is human. To really foul up requires a smell checker.