Last year, the Blaster worm hit medical centers and hospitals in Houston, knocking some offline for several days.
"It was nasty, people lost their jobs," said Mark Mollere, a senior security analyst at Methodist Hospital. "We were lucky we were able to keep our systems up and running while we fixed the problem over a nine-day period."
Prompted by that and the growing need to comply with federal mandates imposed by the HIPAA legislation, Methodist Hospital employed the know-how of several software vulnerability patch management companies. The desktop group used Novell's ZENworks Patch Management software, while the server group used Citadel Security Software's Hercules remediation program.
The software gathers information from a variety of vulnerability scanners and presents them to information security staff via a Web browser. "It can be programmed to patch software vulnerabilities automatically and alert staff to other problems such as rogue e-mail servers," said Mollere.
The need for automated patch management and vulnerability remediation programs has become paramount. In 2002 there were 4,137 software vulnerabilities discovered. On average, it takes an hour to manually fix each exploit on each affected machine. You do the math.
Such nuisances are increasing the cost of ownership. One worm, Code Red, cost U.S. corporations in excess of $2.6 billion, according to Computer Economics. Now many companies are relying more heavily on automated vulnerability remediation software products from BigFix, Shavlik Technologies, Ecora Software and PatchLink, which not only detect the problem but also automatically secure the system with little or no human intervention.
"Worms like Slammer and Blaster were the straws that broke the camel's back because companies simply could not keep up with the volume," said Mark Nicolett, an analyst at the Gartner Group. "So they are either using distribution software packages, they are employing specific software patching systems or they're hiring third-party consultancies to manage the process."
However, these days automated patch management is only part of the solution. "You need to deal with misconfiguration, unnecessary servers and weaknesses such as 'admin' user names and passwords," said Carl Banzhof, CTO of Dallas-based Citadel Security Software.
Other companies provide a blend of consultancy and software services. SecureInfo Corp. in San Antonio keeps a large database of viruses and worms, and its analysts alert customers, including the Department of Homeland Security and the US military, to new vulnerabilities. It uses a tailored version of BigFix's enterprise vulnerability remediation software to push out tested patches to its customers.
However, in addition to the increase of worms and viruses, brought about in part by the use of automated hacking tools such as Netcat, the time between the discovery of the vulnerability and the release of the exploit is shrinking.
Last year it took 26 days between the discovery of the vulnerability and the release of the Blaster worm that exploited it, but last month it took just 36 hours between the discovery of a vulnerability in Internet Security Systems' RealSecure and BlackIce firewall software and the release of the Witty worm, which took advantage of the flaws.
That's why some companies find it desirable to outsource the entire process to companies like Foundstone Inc. in Mission Viejo, Calif. "What's worse is that the Witty worm was one of the first really destructive exploits that crashed hard disks," said Dave Cole, Foundstone's vice president of product management. "Who can afford to wait until their security team can manually patch each machine?"