Manage Learn to apply best practices and optimize your operations.

Creating a proactive enterprise security incident response program

Every organization should develop a proactive security incident response program to ensure that when an incident does occur, it can be handled quickly and efficiently. Contributor Marcos Christodonte II explains how.

Information security incidents are a fact of life. We have witnessed them on the news and within our own organizations -- attackers are getting into networks and stealing corporate secrets and customer data. It's vital to take a proactive approach to incident response to be sure certain enterprises are equipped and ready for the next incident.

Incident preparation helps enterprises maintain controlled and efficient responses during chaotic incident response moments. While the ideal scenario would involve companies avoiding incidents altogether, it's important to be realistic and make preparations that will allow for a brisk response in the event of a security incident. There are numerous steps to take in preparation, and in this tip, I outline several necessary steps for creating an efficient security incident response program.

Establish an incident response program and team
A security incident response team should set up a security incident response program in order to ensure there is an organized, management-approved plan in place. Management must support the team, and sign-off on the official charter, which should outline the mission, scope and objectives of the incident response program. Along with technical incident handlers, the incident response team should include operational and administrative support from IT, legal, HR and management.

Additionally, the security incident response team should work with management to set pre-defined thresholds for leaving systems infected for forensic analysis, isolating systems in "incident" VLANs, or completely disconnecting infected systems. It's critical to be able to disconnect critical systems when dangerous situations occur, such as data exfiltration; it may be disruptive, but it's better to disable a critical system as a last resort containment strategy, than to watch sensitive data fly out the door.

Once the plan is developed, the incident response team should create incident scenarios, test and practice pre-planned responses on an ongoing basis. This means providing resources for training response staff as well.

Establish a known good baseline
Detecting specific attack events -- what files were accessed, deleted, added or modified, for example -- is easier when enterprises already know what is on their networks. Establishing a known good baseline involves performing an inventory of all critical system files and obtaining their MD5 (Message-Digest algorithm 5) checksums. An MD5 checksum is a hash value of a file to perform validation against after an incident occurs. This enables responders to validate the integrity of files to see if any changes were made. For checking critical Windows files, a free utility called MD5 Deep can perform recursive checksum calculations in directories and subdirectories. For Unix and Linux flavors, the MD5sum command (e.g., # md5sum ) can be used. Other areas to baseline include network traffic, processes and services.

If having personnel watch logs all day is impossible due to limited resources or otherwise, administrators should dedicate time to reviewing logs on a regular basis. To make reviewing and correlating easier, all logs should be centralized to a syslog server and time should be synchronized across the enterprise using an NTP server. Place the syslog server on a separate segment (away from primary servers), with ACLs blocking all but syslog traffic to that server, typically UDP/514. In addition, log management tools such as ArcSight, Tenable Network Security's Log Correlation Engine, and LogLogic can help make log analysis much easier. By reviewing server, application and infrastructure device logs regularly, analysts will develop a better understanding of what's considered "normal" network traffic. The same is true of processes and services. Having a known good baseline will enable analysts to discern when new processes or services are introduced into systems.

Develop an incident response program toolkit
A combination of software and hardware tools can make responding to an incident easier and more efficient. A readily available bag should house all of these tools in a convenient spot for when an incident must be responded to on a moment's notice. That bag should contain cables (cross-over, straight-through and console, just to name a few ), a network tap to passively collect data, blank CDs and hard drives, a laptop, pens and pads. Some good open source tools include Helix (a Linux bootable disc for system analysis), the dd copying program for making binary backups, netcat and cryptcat for moving data across the network, and the Sleuth Kit forensics software. Some commercial forensics tools include EnCase by Guidance Software Inc. and FTK 3 by AccessData Corp. Other useful open source tools include Fport by Foundstone Inc., Sysinternals by Microsoft and Mandiant Corp.'s Memoryze.

Develop security incident response program cheat sheets
When the pressure is on and chaos increases, it may be difficult to remember all the details for an efficient security incident response, especially with an audience of executives and other concerned stakeholders. Cheat sheets can help ease this pressure. A security incident cheat sheet should contain the proper commands and command switches that aren't used on a daily basis, the questions to ask administrators upon arrival at the scene, and the proper steps for forensic evidence preservation. Security expert Lenny Zeltser and SANS have some excellent cheat sheets to get started.

The above steps touch on a few of the necessary activities for efficient security incident response preparation. However, no plan should remain static for too long. Organizations must be proactive in developing new countermeasures and response actions to emerging threats. Start collaborating and sharing incident information with both your organizaton's ISPs and law enforcement. Develop escalation and notification procedures. It's essential to stay current and track attack trends -- especially those within the company's sector. Preparation ensures that responses are planned, understood and executed properly by all members of the organization.

About the author:
Marcos Christodonte II, MBA, CISSP is an information security professional working for a consulting firm. He is the author of Cyber Within: A Security Awareness Story (and guide) for Employees, and former incident handler for NATO. Christodonte can be reached at his website:


This was last published in March 2010

Dig Deeper on Information Security Incident Response-Information