Providence Health & Services
Published: 01 Apr 2011
Most IT leaders know they must align with the business to be successful. However, one area where organizations continue to slip up is incident response planning. Enterprises spend copious amounts of time developing security policies and processes in order to secure systems and prevent breaches and data loss. Yet when a security breach occurs, they typically don't have a process in place to manage a coordinated response, within IT and external to IT.
There are a variety of reasons organizations may not have a coordinated security incident response plan. Perhaps the organization has invested in IT security, which is about technology, rather than information security, which focuses on strategy and business process. Part of that business component is incident handling. While incident handling is tactical in nature, it's driven out of program strategy and is used to manage tactical responses.
SECURITY INCIDENT HANDLING DEFINED
Information security incident handling is an action plan for dealing with intrusions (internal/external), cybercrime (copyright violations, hate crimes, child pornography etc), disclosure of sensitive information or denial-of-service attacks.
For those of you who are familiar with the Information Technology Infrastructure Library (ITIL), you may wonder what the difference is between ITIL's incident management and information security's incident handling. A fine distinction exists in the fact that ITIL does not prepare an organization to deal with events that may result in litigation. Additionally, where the root cause of an event in IT service management may be shared with customers, events that require incident handling are typically confidential and considered "need-to-know." The final distinction lies in the difference between an event vs. an incident. ITIL defines any event that causes business disruption to customers or end users as an incident. In information security, events that require incident handling are related to security.
If your organization has implemented ITIL, work to integrate the concepts of information security incident handling within your incident management process. Beyond promoting a common response for all events that disrupt services, there is also the opportunity to embed security events as a normal part of business operations. In other words, it can act as a diffuser for you and your staff in that you are not the lone purveyors of seemingly bad news to the business. It is accepted that security incidents will occur and the appropriate rigor is in place for management.
Security incidents are unexpected regular events -- the computing world's oxymoron. Any organization with an Internet connection or that uses computers can expect to experience unexpected security events on a regular basis. Why? There are always vulnerabilities found and mistakes made; the regularity occurs with each release of software and each action taken by a technology user.
Due to the unexpected and stressful nature of security related events, be prepared to devote time and attention to the management of the incident. Detailed forms must be completed and descriptive notes taken. Likely, you will identify different types of security incidents, resulting in specific procedures for each type of incident. However, it's unlikely your employer has the capacity to employ individuals primarily for incident handling. Competing responsibilities reduce incident handling at its best to an ad hoc maturity level. Even with training and the right forms and procedures to guide you, it can be challenging to capture information correctly or follow instructions. Cooperation beyond the doors of IT is required.
Incident handling is not an IT process, but a business process that must be supported by the legal, human resources, communications and physical security teams of your organization. Relationships must be built with these teams and expectations set in order to avoid conflict. Information security may drive the leadership of incident handling and provide the necessary training; however, a governance board of sorts should be assembled to oversee the process. Legal and HR must be involved to validate the legal soundness of your program to ensure non-reputation of evidentiary data, which is a costly process.
Best-of-class technical environments are required to establish non-repudiation of evidentiary data. Dedicated hardware and specialized software is required along with training on the use of the software. Combined with the possibility of infrequent use, your business partners -- the legal, HR and other teams -- may challenge cost effectiveness. Be prepared to provide transparency around the cost of people, process, and technology. For organizations with regulatory obligations, the long-term investment should be part of the ROI strategy for selling incident handling. If your organization does not have regulatory obligations, brand protection and reputation are legitimate considerations.
Discretion is an important element of information security incident handling. In ITIL's incident management process, outages and their root causes are shared with customers and business partners openly. In contrast, information security incident handling requires the discretion of all involved parties. Casual bandying of security incidents among employees can reduce the trust of internal business customers. -- More importantly, should the information find its way to customers or partners external to the organization, damage to brand, reputation and trust will occur. Remember the principle of need-to-know (N2K) when handling incidents. Never talk to the press unless directed by your legal team and senior leadership. Also, you can mitigate organizational "chatty Cathys" by providing ongoing education and training.
Review all policies related to appropriate use and technology. While you may not directly write policy, you should familiarize yourself with your organization's policies. Suggest additional policies supporting your incident handling plan. For example, work with the IT team to implement standards that will support the process. Some of the most seemingly granular discrepancies, such as overlooking clock synchronization or log retention policies, can undermine the information you've gathered.
Avoid these common incident response blunders
Do not allow mistakes to derail your incident handling program. Below are some of the most common and costly mistakes that undermine incident handling.
- Disorganization: Incident handling is useless if you do not have a plan or the appropriate forms for recording incident information. Waiting to develop a plan once an incident has occurred will result in disaster.
- Rushing incident resolution: When an incident has occurred, it's important to dedicate adequate time to understand how the incident occurred in the first place. If the investigation is rushed, it may result in missing critical information or reintroducing a compromised system into your infrastructure.
- Lone Ranger syndrome: Relying on your skills and expertise without engaging others. Depending on the maturity of your organization, you may have to champion incident handling, but it's better to at least have the framework for a team and share that with proposed team members than to mishandle an incident and be left holding the bag.
- Discounting Johnnie Cochran: If we assume that we will end up in court, then you should also assume that the court will want to understand how you collected your information. Litigation is all about proving or disproving a person's information. When you cannot provide documentation that would be expected of our industry, then all your handling may have been useless, along with your legal counsel and public relations.
- Violation of need-to-know: Anyone who knows about the incident may have to testify in court. Stories vary in times of stress or through misinformation. Only those directly required to have knowledge of an incident should be briefed. Even then, tell them only what they need to know. Let your legal team determine whom outside of IT should be apprised.
- Tunnel vision: Incident handling is more than just a detailed task; it's a business process. Take a strategic view when approaching incident handling.
-- RAVILA HELEN WHITE
BUILDING OUT YOUR PLAN
Different types of incidents require different types of handling. Determining the response plan prior to an incident will streamline handling. Identify what types of incidents you may have to handle:
- Malware breach and containment
- Information disclosure
- Employee investigations
- DDOS attacks
- Hostile take-over
This may also be the opportunity to scope communication; knowing who needs to know in advance will help in avoiding unnecessary disclosure. For example, a DDoS attack does not require need-to-know (N2K). Most likely, sharing the cause of a network interruption with everyone will not damage brand or reputation.
Determining response plans prior to an incident will also help you pick the appropriate tools for recovering information. Identify what types of technology you may have to handle through technology scoping:
- Client PC technology
- PDA technology
- Cell phones
- Server technology
- Infrastructure technology
Develop forms that will help you capture required information during an incident; SANS has a number of sample incident handling forms. Identify what types of documentation you will need to handle an incident through documentation scoping:
- Contact log -- names of each point of contact during the incident.
- Evidence log -- data points of information collected during the incident.
- Chain-of-Custody log -- audit trail of collected evidence when it changes hands.
- Communication log -- information about incident and contact during initial incident notification.
- Sanitation record -- information required for sanitized media.
- Recovery record -- steps taken to recover from incident.
- Eradication log -- steps taken to remove malware or hostile user.
- Identification record -- type of incident and who discovered it.
PREPARE A TOOLKIT
Incidents can occur any time. Have a toolkit prepared allowing access to all documentation required to capture incident information from beginning to end. When network compromises occur, you may not have the luxury of retrieving electronic documentation from the network. Having printed documentation ensures the information you capture is as accurate as possible. You must have the same documentation at home. Incidents are not considerate; they may happen at 3 a.m..
When possible, request dedicated technical resources, such as a spare drive for imaging and a spare workstation for gathering logs and performing other forensics tasks for incident handling. It is important to establish non-repudiation of the information you've gathered. Even if you cannot justify a dedicated server for storing recovered information, request storage that has limited access. Require periodic access audit of the storage area controlled by key card access to establish confidentiality enforcement.
When handling incidents that require physical evidence recovery, store the hardware in a location that is secure, with limited access. You must also consider what collection tools you will use, such as log file analyzers, disk imaging, and forensics software. Select commercial tools for information capture. Open source tools are cool, but organizations should purchase commercial tools to establish non-repudiation. If you do choose to use open source tools, clearly document them as part of your toolkit.
Incident response training pays off quickly
Consider investing in training for at least one person in your organization. The dollars spent are easily returned the first time you must respond to a security breach.
The SANS Institute has a great reputation for providing incident handling training. Certification as a SANS GIAC Certified Incident Handler is certainly a step in the right direction to ensure your organization can recover from a security breach or respond adequately to a litigious event.
There are legal implications related to incident handling. Therefore, remember that whether you are certified or not as an incident handler and receive training, you need to partner with your legal counsel to implement and, if necessary, supplement the organization's incident handling program.
--RAVILA HELEN WHITE
NOTE-TAKING BEST PRACTICES
Take good notes, but keep in mind that lengthy notes containing unnecessary information are not helpful and will only muddy the incident should it result in litigation.Notes must be informative and concise and contain the facts of what you are handling. For instance, it is not necessary to mention that you are dealing with a suspected embezzler/child pornographer/information thief, etc.; this is where your forms will assist. Complete generic fields prior to handling an incident. Record only those facts related directly to your evidence recovery:
- Who -- contacted you; performed information recovery
- What -- was recovered (e.g. log files)
- When -- date and time of recovery
- Where -- location of recovered evidence
- How -- tools and method for evidence recovery
POINTS TO KEEP IN MIND
Any incident involving the recovery of information or technology associated with employee misconduct should be reported to your resident legal counsel. They will make the call regarding the appropriate actions to take. You should also notify legal if a third party who's been entrusted with your data experiences a breach resulting in exposure.
Remember, incident handling is a business process which requires a plan. You will need a thorough understanding of your organization to propose and implement a systemic plan. Once a plan is in place, update the process regularly. Just as disaster recovery plans and backups should be tested, so too should incident handling procedures. This will ensure the kinks are worked out prior to an incident.
Understand your responsibility. Incident handlers get into trouble when they go beyond the bounds of what is appropriate for their role. Only ask questions related to capturing relevant information. Beware of acting as a proxy to HR and legal when it comes to dealing with people. Do not go beyond the role of an incident handler. It's more trouble than it's worth.
Ravila Helen White is the director of enterprise security and architecture at a company in the Pacific Northwest. Prior to that, she was the head of information security at The Bill & Melinda Gates Foundation and drugstore.com. Send comments on this article to email@example.com.