St. Patrick’s Day will never be the same for executives at RSA.
In the world of information security, March 17, 2011 was truly a day of infamy, especially for the millions of end users who trusted the integrity of the ubiquitous RSA SecurID authentication token. On that day, RSA executives had to deliver the announcement they’d likely only imagined in cold-sweat nightmares. One year ago on March 17, RSA executives had to tell the world their flagship intellectual property had been stolen, likely by a well-financed, determined and patient nation-state adversary.
One of the world’s best-known security companies had been compromised; RSA, the Security Division of EMC, now was in the shoes of many of its customers. It also ran the risk of becoming an industry punch line. But this was no ChoicePoint situation. They weren’t talking about personal data of 117,000 individuals. This was mission-critical, two-factor authentication technology prevalent in the financial services industry, U.S. government agencies and the country's defense industrial base. Little did RSA know that last March the SecurID intrusion and exfiltration of IP was just a hop in a bigger attack on the defense industrial base – less than two months later, Lockheed Martin emerged as the target of a thwarted attack downstream of the RSA breach.
In the months following the attack, RSA bought eight robots to increase manufacturing seven-fold to meet customer demand for replacement tokens and stave off damage to its strong brand within the industry. Meanwhile, customers scrambled to replace tokens – no small feat for a large organization – and rethink their security processes. The RSA breach gave its competitors a window of opportunity as some organizations looked to alternative authentication technologies. Most of all, the breach was a wake-up call: If RSA could get hacked, anyone could. If anything underscored the need for defense in depth, this was it.
“It was a huge blow to the security product industry because RSA was such an icon,” says Jennifer Bayuk, an independent information security consultant and professor at Stevens Institute of Technology. “They’re the quintessential security vendor. … For them to be a point of vulnerability was a real shocker. I don’t think anyone’s gotten over that.”
A LIVING HELL
For RSA, the days immediately following the discovery that an intruder had penetrated the company’s systems were some of its darkest.
“It was hell to live through what we did,” RSA President Tom Heiser said at a press event in January. “We had absolutely flat decision making; there was no hierarchical decision making. We needed to figure out how to get out of this, because we were getting pummeled.”
RSA’s immediate focus was to not only triage the attack, but to be proactive about telling customers what had happened.
“We had to stop the bleeding and deal with customers,” Heiser said. “We decided within 21 hours of understanding that customers may be compromised; we informed them. Ahead of our partners too; we didn’t give our partners a chance of communicating with their customers. There was no debate to do this.”
At the press event, RSA Chief Executive Art Coviello said investigators learned the initial attack started at a third-party, setting the stage for cybercriminals to design a targeted, social engineering attack against RSA employees. Using a spear phishing campaign, the attackers lured an employee into retrieving a message from a junk mail folder and opening a Microsoft Excel spreadsheet containing an Adobe Flash zero-day vulnerability. From there, the attackers targeted other systems, elevating their privileges until they could gain access to RSA’s proprietary data.
Coviello wrote a letter to customers in which he labeled the attack on SecurID an APT and said “certain information” had been exfiltrated from RSA’s systems, some of it related to SecurID.
“We believed we were attacked for the purposes of getting to the country’s government and industrial base,” Coviello told reporters in January. “We believed we had a very strong security system in place before the breach and we redoubled our efforts across the entire spectrum, including our communication with employees.”
The company increased its manufacturing capability seven fold, engaged its largest customers to explain the attack in detail and later took its story on the road in nearly two dozen advanced threat summits held around the world. The security division of EMC Corp. reported the breach cost $63 million in initial expenses.
“We worked to restore confidence via direct discussions with customers, technology and services,” Coviello said. “We knew a lot about APT, and yes we fell victim, but many looked at this as a watershed moment. It happened to RSA, it could happen to anyone. Our job was to create a security infrastructure that shrinks the window of vulnerability so no material damage is done.”
While some customers say RSA was quick to inform them about the attack, others say they were left searching for details. The initial information RSA released publicly left many security professionals and industry experts scratching their heads, wondering what exactly had happened.
“Their response to the incident was complicated. There was a lot they probably would have disclosed and there’s probably a number of reasons they could not have disclosed it,” says Scott Crawford, managing research director of security and risk at Enterprise Management Associates, an IT industry analyst firm based in Boulder, Colo. “Anyone who has worked in security and faced a situation like that probably knows what I’m talking about.
“They did make an effort to reach out to their most affected customers, but not everyone was pleased with the outcome of that,” he adds.
Chris Ipsen, CISO for the state of Nevada, says RSA was proactive in reaching out to him. “They called me immediately and said, ‘We’ve got a problem.’” He and others in the industry give RSA credit for being open about the breach. But Scott Greaux, product manager at PhishMe, a Chantilly, Va.-based software firm focused on training users how to avoid phishing attacks, describes RSA’s communication strategy as “disastrous.”
“I don’t think they had a playbook for this type of event,” says Greaux, who was working as deputy CISO for a Fortune 50 company at the time of the breach. “It’s probably a good lesson for any firm. … You have to have some type of disaster communication plan. It was very frustrating trying to get good, valid information. You heard snippets. It was hard to vet what was true and what wasn’t.”
An RSA spokesman says within the first month of the attack, the company “executed a massive outreach program reaching more than 60,000 customers” with its online support system notes, talking to more than 15,000 customers by phone and addressing more than 5,000 customers via conference calls and face-to-face meetings.
For small businesses with 25 users, replacing SecureID tokens was a fairly straightforward task, Greaux says. But for large enterprises with multiple offices branches and offices, the effort was substantial, requiring a lot of communication, coordination and shipping. Greaux says a conservative estimate of the cost for large companies was $12 per token.
“There’s the hard cost to overnight a token and the soft cost of taking resources away from perhaps more critical business projects to focus on what is usually a mundane task of replacing a token here and there for users,” he says.
On the positive side, the breach led organizations to take a close look at how they use RSA SecurID and how they could improve their processes to ensure they’re protected with RSA or any multifactor authentication technology, Greaux says.
At SAP, where a large number of employees use SecureID for remote network access, the attack on RSA led the company to invest in an initiative to modernize its authentication infrastructure earlier than originally planned, according to Ralph Salomon, head of SAP’s global IT security and risk office. Soon after the incident, the global enterprise software giant decided to replace all its SecurID tokens and at the same time, start on the modernization effort by replacing a large number of hardware tokens with software tokens, he says.
“By moving to software tokens we’ve been able to circumvent/avoid the production bottlenecks of the hardware tokens at RSA caused by the sudden increase in demand,” Salomon says. Plus, employees benefit “from an improved ease of use and have one less device to carry around,” he adds.
The authentication provided by SecurID is important, but not the only security measure SAP has in place for remote connectivity to its networks, Solomon says. “In addition, we are currently introducing an authentication methodology on a network level, supporting the secure access to our company network,” he says.
Financial Engines, a Palo Alto, Calif.-based independent investment advisor, immediately asked RSA for replacement tokens and also added “additional monitors to track any unusual activity around authentication with our tokens,” says Matthew Todd, CSO and vice president of risk and technical operations.
Increased monitoring was particularly pervasive in the defense industrial base -- the apparent target of the RSA attackers. “Everyone has priorities for what they monitor and something like two-factor authentication wasn’t high on the list, but now it is,” Bayuk says. “People are changing their operational strategies. Companies that are very security aware, like the defense industrial base, are changing their strategy.”
The RSA breach shook the faith of many firms, Todd says.
“Before the breach, saying you use two-factor authentication was a sure-fire way to satisfy an auditor. After the breach, IT teams were left wondering just how secure the two-factor process was,” he says. “There was a lingering sense that even new tokens were potentially tainted somehow.”
COMPETITORS MOVE IN
The attack hit RSA’s reputation hard, says Avivah Litan, a vice president and distinguished analyst at Gartner.
“You’re only as good as your reputation in the security business, and they’re kind of living with this skeleton in the closet,” she says. “With customers, whenever you mention SecurID, the first thing that comes into people’s heads is the breach.”
While the attack impacted RSA’s reputation, it didn’t lead to a mass exodus of customers; it would be hard for an enterprise that’s invested in SecurID to switch out the infrastructure, she says. Nonetheless, the breach gave RSA’s competitors an opening. For organizations looking for an authentication solution and don’t already have RSA, “RSA isn’t at the top of the list like it used to be,” Litan says.
She also heard from many customers angry about the licensing terms that came with the replacement tokens. “There were a lot of strings attached,” she says. An RSA spokesman says the company’s remediation program was “designed to be fair and equitable” for customers. “RSA did not ask customers to commit to additional spending in order to receive replacement RSA SecurID tokens,” he says.
Litan says the attack has been a boon to alternative authentication technologies, such as phone-based authentication, she says. Banks that look to provide secure access to their external customers are increasingly moving to phone-based authentication. “The banking industry has realized tokens aren’t the solution, not just because of this [the breach] but because of man-in-the-browser Trojans,” Litan says.
Enterprises wanting to provide secure access to their employees are putting more of an emphasis on risk-based authentication and software tokens, she says, noting that RSA provides these other methodologies as well. “It’s a natural evolution,” Litan says. “People have been moving away from token-only, one-time password solutions.”
Competitors say they saw a surge in inquiries about authentication options such as software tokens and smartcards but they’re generally circumspect when talking about the RSA breach.
CA Technologies, which offered RSA SecurID customers a program to trade in their tokens for CA ArcotID software credentials in the wake of the breach, got a lot of inquiries but can’t disclose actual figures, says Lina Liberti, vice president of security product management at CA Technologies.
“Tokens have always been viewed as the most secure type of solution you could put in place for advanced authentication,” she says. “From an industry perspective, it was a bit tarnished.”
At the same time customers were calling competitors to find out what their options were, they now also had a lot of questions about the security around the authentication product itself. “They started to ask questions I’d never heard before, such as ‘How are the seed numbers secured?’” says Tsion Gonen, corporate vice president of products and marketing at SafeNet, a supplier of authentication and encryption technology.
“What the breach did was help customers understand that they really need to look at the overall solution when they’re looking at security, especially when it comes to authentication and there are keys and processes in play,” Liberti says.
It also led customers – whether they decided to stick with hardware tokens or use an alternative authentication technology – to take a layered approach and build in additional authentication, such as device identification, she says.
Gonen says organizations were already rethinking their authentication strategies when RSA was attacked. One-time password token technology has been around for years and hadn’t evolved much, he says: “People were thinking about how the threat vectors had changed and the way they need to change their infrastructure.”
For its part, RSA says it had no meaningful customer attrition directly related to the breach. The attack served as something of a wake-up call to reinvigorate SecurID; at the press event in January, executives talked about introducing a software development kit that gives banks the ability to build SecurID into mobile banking applications. SecurID can be used as a mobile app, enabling employees to use their smartphone to authenticate, they said. The company also is looking at the mobile platform as a way of incorporating geolocation data and biometrics into the authentication process.
SAP’s Salomon says RSA not only had to deal with emotional customer reactions but the logistical challenge of replacing a large amount of tokens in a short time while increasing the company’s overall security level. “In our eyes, RSA managed these challenges very well under the given circumstances,” he says.
While the attack on RSA shook the industry to its core, it had an upside by driving home the need for layered security, security professionals and industry experts say.
“In terms of ‘silver linings,’ at least this served as a wake-up call for defense in depth,” Todd of Financial Engines says. “As more details of the breach came to light, and the sophistication – and downstream target – was revealed, many firms started to discuss the threat of APT and the reality of highly sophisticated attackers with deep pockets.”
Nevada’s Ipsen says the breach highlighted the need for best practices and a multi-layered approach to security that takes the human element into account.
“It’s increased awareness around the vulnerabilities in the technologies we rely on,” he says. “It’s imperative that we constantly improve our practices and don’t overly rely on any one technology.”
Organizations are looking at ways to improve their security strategies beyond techniques for strong authentication so they can better defend themselves against targeted attacks, EMA’s Crawford says. “Added intelligence is part of the arsenal they’re looking at overall,” he says.
RSA is trying to meet the demand for increased intelligence. The company is working to integrate the network monitoring technology from NetWitness, which it acquired in April, with its Archer compliance management platform. The goal is to provide integrated technologies that give customers better visibility into their infrastructure.
The company unveiled its first integration piece last year in the form of NetWitness Panorama. The connector tool combines threat intelligence with log data. It helps security analysts drill down into network threats by joining NetWitness’ data and analytics to RSA Envision SIM log collection capabilities. Panorama supports 200 different enterprise log formats putting threats into more context.
“Security has to be more intelligence-based and positioned to understand that we’re living in an environment of advanced threats,” Coviello told reporters in January. “It’s not a matter of if and when, it’s how you are able to respond and shrink the window of opportunity so when you are breached you can respond timely enough to mitigate any damage.”
In fact, the attack on RSA – and all the other breaches that have surfaced in recent months – led companies to focus on “post-breach security,” says SafeNet’s Gonen. Security technologies have traditionally focused on preventing breaches, but organizations are now concerned with implementing measures that protect corporate assets from attackers that have already broken in, he says.
“People understand the problem,” says Philippe Courtot, chairman and CEO of Qualys, a provider of vulnerability management services. “CSOs are worried about themselves. … They’re thinking, ‘I may already be compromised and not know it.’”
RSA, the Security Division of EMC, is just one of several security vendors that have been compromised in recent months. Here are some attacks that have surfaced over the past year:
Symantec – In January, the company revealed its systems were breached in 2006, resulting in the source code leak of its pcAnywhere program as well as certain outdated software. The incident forced Symantec to advise customers to disable pcAnywhere until vulnerabilities were remediated.
VeriSign – In a U.S. Securities and Exchange Commission filing in October, VeriSign reported that attackers penetrated its network infrastructure several times in 2010. The company said attackers potentially obtained data from a small portion of its computers, but it didn’t believe attackers breached the servers that support its Domain Name System (DNS) network.
Stratfor – Security intelligence consultancy Strafor took its website offline following a Christmas Eve attack in which intruders penetrated the website and stole thousands of credit card numbers. An Anonymous hactivist group claimed responsibility for the attack. Strafor said attackers stole data associated with organizations that purchased its publications.
DigiNotar – In September, The Dutch certificate authority, a subsidiary of VASCO Data Security, declared bankruptcy in the wake of a July 19 breach of its digital certificate systems. Hackers acquired a digital certificate from DigiNotar, which enabled them to issue fake public key certificate requests to a number of domains, including websites owned by Google.
Comodo – The issuer of SSL certificate warned customers last March that it issued fraudulent certificates to seven Web domains including Google and Yahoo after criminals compromised one of its partners.
HBGary Federal – Last February, attackers associated with Anonymous broke into HBGary Federal, stealing research and other data associated with HBGary co-founder and malware expert Greg Hoglund.
Marcia Savage is editor of Information Security magazine. Send comments on this article to firstname.lastname@example.org. Michael S. Mimoso is editorial director of the Security Media Group at TechTarget. Robert Westervelt is the news director of SearchSecurity.com.
- Data Breach: Security Investigation, Detection and Rapid Response –Splunk
- Managing data breaches –ComputerWeekly.com
- Data Breaches in Europe –ComputerWeekly.com
- Essential guide to dealing with a data breach –ComputerWeekly.com