Patching used to be a real drag for Gabriel Selmi, the security designate for non-profit mental health services provider Advanced Behavioral Health Inc. of Middletown, Conn. When an update arrived, the network administrator and his tiny IT team would download it to a floppy disk and then walk around to about 50 machines. Or, they'd send out an e-mail with a link and ask the employees to do it.
"It was a complete nightmare for us, a lot of manual work," Selmi recalled.
Today, Advanced Behavioral Health's all-Windows shop now supports 200 local PCs at its headquarters and another 150 remote users that dial into the network using a VPN. But patching is no longer problematic, even with the window between a bulletin's release and exploit's circulation narrowing.
In 2004, and after months of serious comparison shopping, Selmi settled on a patch and vulnerability management service from Scottsdale, Arizona-based Patchlink Corp. that has eliminated much of the manual labor. But the patch landscape for many others remains riddled with land mines, and some enterprises are rushing to seal holes with unofficial patches or before properly testing sanctioned ones.
Proving that point, Patchlink on Monday released the results of a new customer survey that asked more than 250 CIOs, CSOs, IT managers and network administrators about their patch management practices. The results are based on information gathered during the company's 360 Security Conference in Tempe, Arizona, in February.
Among the results:
How much is too much
Patch cycles remain controversial, particularly among users of major vendors like Microsoft and Oracle Corp. Both have been accused of leaving customers at risk by delaying the release of critical updates. While the two vendors staunchly defend their actions, security researchers have long criticized them for not going public soon enough when serious flaws in their software programs are discovered. Increasingly, others are issuing independent workarounds or patches ahead of a vendor's official fix.
In December, when malware writers found a way to embed malicious code in Web images using the Windows Meta File flaw, reputable organizations like the SANS Internet Storm Center encouraged security pros to download researcher Ilfak Guilfanov's API block in the absence of anything official out of Redmond.
Then, just last week vendors eEye Digital Inc. and Determina Inc., both located in California, issued actual binary patches to the DLL for the createTextRange flaw in Internet Explorer, which Microsoft has yet to patch.
"They may be from reputable sources, but if they aren't released by Microsoft and approved by Patchlink… We don't bother with third-party patches," Selmi said. The network admin said such faith in third parties remains dangerous, "and I don't see that changing any time soon," he said.
Selmi is not alone, according to the Patchlink survey. Seventy percent passed on Guilfanov's solution and waited until Patchlink had vetted the approved Microsoft WMF patch, which ended up being released ahead of Patch Tuesday due to public pressure.
Chris Andrews, vice president of security technologies for Patchlink, said those enterprise IT shops still tending to patches themselves should be especially suspicious of fixes that arrive in unsolicited e-mails.
"There are a lot of folks getting involved in this process, with security companies starting to put out temporary fixes," Andrews said. "The last thing you want to do is get a patch from an untrusted source. We've all seen the e-mails claiming to have new fixes for new Microsoft issues and suddenly your users are trying to take security into their own hands and next thing you know you've got spyware all over your network."
He recommended a staged approach. Test the patch with a small sample initially (off- and then online) and if successful, move to a pilot group and then the general network population. "You need to take a cautious approach and not just blast the patch straight out, you could run into problems. It's always best to test on a small number of representative machines first."