Three security managers in the Gulf Coast tell their stories.
Crises require information security officers be the voice of reason and Hurricane Katrina truly tested the mettle of security managers on the Gulf Coast. It caused immeasurable distress as it leveled businesses, keeping companies offline for weeks while power and telecommunications were restored. Corporate mandates to get operational quickly threatened to make security another casualty. And, cyber-attackers showed little mercy to devastated businesses, often targeting those the storm hit hardest.
Information Security recently spoke to three security managers who survived Katrina. They shared their insights, talked about how they plan to rebuild and imparted the lessons they learned.
Once the storm passed, what security issues surfaced that you did not anticipate?
Jennifer Creger: Any time an office is moved or converted into a temporary facility, there are going to be security issues. We followed our 100-plus page disaster recovery plan that we activated the Friday before Katrina made landfall. Twenty-four employees are part of the recovery team, and they immediately moved to a temporary office in Baton Rouge. As time passed, more employees arrived there. We had real challenges providing security of information systems access because we had more employees than computers.
Tim Ozmun: We watched our firewall logs and saw a script-kiddie hammering away at us. I don't know if it was random or if he'd selectively chosen our site. We don't have classified information, but our information is part of a public trust domain. A lot of our data is used to make determinations. [Huge transactions] are determined by the kind of information we have and how it's presented. We have a public trust that the information is accurate and available, and its integrity is kept.
Bill Derwostyp: We had intensive looting on campus; we know 17 offices were broken into. We also had problems getting physical access to buildings. Once things settle down, we'll have to verify who has machines and whose machines may have been stolen.
How did you keep security from becoming an afterthought in a tragedy?
Ozmun: From an information security standpoint, all of our rules were in place. But in an emergency, there are things you don't normally do that people start to feel is OK to do. [It's my job] to ensure that the risk is still verbalized, and to make sure my superiors and data managers understand [the risk]. Folks tend to get more lax: "Get the job done; it's a big project." It's important to have the voice of the information security officer. [In a disaster] a security officer's job doubles; you have to have more log checks, make sure your IDS is up and running, and watch for patterns you don't normally see.
You also have to be more vigilant. If you need to convince your managers that a particular risk is too great, run it through the NIST formulas and bring it to them. Ask them if this is the kind of risk they want to take, and have them talk to insurance companies to get a qualitative value. Sometimes the risk is OK to take in management's eyes. [In this scenario] security officers have to be flexible. If it's decided to accept the risk, then you need to kick it into high gear on the IDS side of the house. In emergencies, security can be thought of as optional, but that's the worst time to have that frame of mind. There will always be somebody out there lurking and waiting [to take advantage of your vulnerabilities].
Creger: You have to make some adjustments to your [security] program because the top priority is customer care and recovery. But you can do it and still have good security practices.
At first, we were in close quarters. Branch personnel, bookkeepers, human resources, lending and other functions were all handled by one office. The IT department was definitely on top of security issues, not to mention troubleshooting. We had a database of system access privileges that we kept intact, but we also created new groups for Katrina-response employees who were given greater authority for the storm response. However, we know where we were pre-storm, and we will get back to that system.
How involved were you in disaster recovery planning, and how is security treated in the plan?
Creger: I'm on the 24-member team, and our physical security officer is on the team along with others from our IT staff. We certainly considered security. We have an audit trail for all our manual entries while systems were temporarily unavailable due to power outages.
Ozmun: Our business continuity plan has been in place for a while. Security was treated more from a personnel security [standpoint]--making sure our emergency contact lists are up to date and everyone had them. We also had to check our backup status. This usually takes a bit of time. If we have time, we'll do another backup.
Derwostyp: The DR plan was developed in 1999, and a project plan was submitted in August to redevelop the DR and continuity plans. But they weren't acted on [before the hurricane]. Therefore, our continuity plan was rudimentary; there was nothing in the plan, for example, to replace communications in an emergency, nor were there plans for our own emergency response team. This is an opportunity to refine policy and procedures.
Which contingency plans worked best? What improvements would you like to see in future plans?
Creger: Having a plan is essential. The ability to call upon alternative resources is impossible without advanced planning. Within 24 hours of Katrina making landfall, we were able to serve our customers. Telecommunications were, and continue to be, our biggest obstacle. Knowing our critical applications and systems was a key to success.
Ozmun: From an information security perspective, what we could have done better was establish alternatives for Internet connectivity and communications, which was the real problem. We're reliant on a single leased line, so we're looking at additional systems for redundancy. This kind of hurricane activity could be up for years, so we're considering buying mirrored systems and, for sustainability, moving our data out of state. We'll have to purchase a PKI structure of some kind, as well as more VPNs. Homeland Security is requiring us to do more with security; money has to be made available for these things.
Derwostyp: We're working on new policies to tighten our procedures on a university-wide basis, putting all five state colleges under the same umbrella and in compliance with regulations like HIPAA, SOX, GLBA and the Mississippi Public Records Act.
I come from a corporate background, and the academia mindset is they can do anything they want, whenever they want. I need to change [that mindset]. I want to establish a secure process and not open up the university to liability.
I would also like to see a team formed that can respond to incidents and has the power and authority to do what needs to be done--like spend money, bring in equipment and be a best-practices model for the university.
I'd also like to see a separate security policy and compliance team that would report directly to the president of the university and have full authority to manage security on all systems as it relates to federal and state regulations.
Read full versions of these interviews online at SearchSecurity.com/ismag.