Since the announcement of Heartbleed, a serious flaw that could allow an attacker to access passwords, encryption keys and other information sent to a server using OpenSSL, many companies have scrambled to patch their systems and revoke security certificates. While patching hundreds of thousands, if not millions, of systems will take time, the impact of the massive re-registration of SSL certificates and the wide use of revocation could be equally severe.
When content-distribution firm CloudFlare updated its Secure Sockets Layer (SSL) certificates to protect its clients from the impact of the Heartbleed bug, that action alone caused minor palpitations across the Internet.
Roughly 50,000 keys were no longer trustworthy, and the size of the certificate revocation list (CRL) suddenly ballooned by more than a factor of 200 to nearly 4.7 megabytes, overnight. The CRL is one of two ways that browsers can check that an SSL certificate, widely used to secure communications between browsers and websites, has not been compromised. Just the bandwidth costs of distributing the new CRL file to browsers would likely surpass $400,000 dollars, according to CloudFlare's calculations, while a check using Amazon's Web Services put the figure closer to one million dollars.
Certificate authorities (CAs) such as Comodo, Symantec and Trustwave, are currently allowing companies to revoke and re-issue certificates for free. For those CAs, the slow response to Heartbleed by much of the business world is a mixed blessing, according to Comodo Chief Technology Officer Robin Alden. The costs of dealing with the mass revocations will be high, but the uneven response has given certificate authorities a chance to keep up with requests.
"In spite all of the news out there, plenty of our customers are only just starting to respond," Alden said. "It is not a good thing for Internet security as a whole, but at least the fact that they are taking time to respond spreads out the load of re-issuing certificates."
Brian TrzupekVP of managed identity and SSL, Trustwave
With analysis firm Netcraft estimating that some 500,000 sites use versions of OpenSSL that are vulnerable to Heartbleed -- and with many companies using private SSL certificates inside their own networks -- other certificate authorities could face similar floods of requests and burgeoning revocation lists, taxing the certificate infrastructure that underpins much of the Internet's security.
"Beyond the cost, many CAs are not set up to be able to handle this increased load," Matthew Prince, CEO of CloudFlare, stated in an analysis of the costs of mitigating Heartbleed. "Revoking SSL certificates threatens to create a sort of denial-of-service attack on their own infrastructures."
Revocation process lacking authority
Despite playing a critical part in the Internet's reliance on SSL, certificate revocation has uneven support. Before Heartbleed, revocation was an uncommon event and most users rarely encountered a revoked certificate. As the attacks on the Canada Revenue Agency and widespread detection of scans for the Heartbleed vulnerability show, attackers are now using the vulnerability to try and gather information on passwords and certificates.
Those attacks make revocation, and the support for blocking revoked signatures, extremely important, said Brian Trzupek, vice president at security services provider and certificate authority Trustwave.
"The concern is that the revocation mechanisms en masse have not seen this kind of test on them in such quantities," Trzupek said.
When a certificate is compromised, CAs can choose one of two ways to communicate that untrustworthiness to Internet users and browsers. Browsers can either make a request every time they encounter a new certificate – usually when the user goes to a new website -- using the Online Certificate Status Protocol (OCSP) to check whether the certificate has been revoked, or they can occasionally download a copy of the CRL and use the list to determine the trustworthiness of an SSL certificate.
The OCSP method requires less bandwidth for each additional lookup, though potentially more over time, while browsers that utilize CRL require fewer, but larger downloads to make the same certificate check.
Different browser makers use different methods. Google's Chrome, for example, synthesizes its own lists from OCSP and CRL information, while other browsers like Mozilla's Firefox have stopped using CRLs altogether. Regardless of which method browsers ultimately rely on, the sheer amount of revoked certificates in recent weeks -- the SANS Institute's Internet Storm Center pegged the increase in recent revocations at somewhere between 300% and %500 -- is creating a wave of traffic that is likely to cause issues, particularly for mobile devices with less processing power and memory than traditional PCs.
"All the sudden you are going to have something on a wireless or cellular connection downloading, not a 2 kilobyte CRL, but something that is 3 or 4 megabytes," said Trustwave's Trzupek. "And that is going to put a lot of strain on these devices."
Even worse, the revocation process may not be delivering that much in the way of security benefits for many users, according to Michael Klieman, senior director of product management for Symantec's Trust Services team, because most browsers do not prevent a user from accessing a site with a bad, or revoked, certificate.
"Blocking user access to websites is not in the browser makers' interest," Klieman said, "but from a security standpoint, it has to be done to protect users."
Heartbleed's long tail
Unfortunately, the issues surrounding Heartbleed are unlikely to be resolved soon. While security firms have urged companies to change their keys, Trzupek said the OpenSSL vulnerability will likely remain unpatched on many platforms. Some systems, such as Web servers using the secure HTTP protocol (HTTPS), are obviously vulnerable, but so are a number of less visible Internet platforms, including mail servers and proxy servers. Millions of mobile devices running the Jellybean 4.1.1 version of Google's Android operating system may also be vulnerable, though the practicality of Heartbleed-based attacks against such devices remains unclear.
Kevin Bocek, vice president of security strategy and threat intelligence for key management technology vendor Venafi, emphasized that business can't address the problem of potentially compromised keys until after they've patched their systems, a reality he said many companies still do not understand.
"There have been a lot of misperceptions," Bocek said. "People believe that they just need to patch public-facing systems, while some feel that all they need to do is reissue certificates."
As part of the Heartbleed clean-up process, Bocek advised companies to uncover all of the systems where SSL keys may be used for security, prioritize the patches based on the criticality of each system and, once updated, create new keys for the systems. Even private CAs, used inside many large corporations to secure internal access to servers at a lower cost, should be patched and re-keyed, Bocek noted. Otherwise, any attacker that gains some level of access to systems inside a company will easily be able to compromise the entire network.
"It is the last-mile problem," Bocek said. "You have keys and certificates used across all these applications, but you don't know where they are used."