It wasn't that long ago that patching was an all-hands-on-deck emergency. At the first inkling that Microsoft (or any vendor) was releasing a fix, security managers would muster their staffs to begin the arduous process of testing, installing and retesting vulnerable machines.
Even with the utmost diligence, getting to every machine was next to impossible. In the worst cases--such as the first three patches for the MSDE vulnerability exploited by the Slammer worm--the patch either didn't work or undid previous fixes.
For years, we've heard enterprises bemoan the lack of reliable automated patching tools and vendor support. Worse, the poor quality of patches--particularly from Microsoft--has opened new vulnerabilities or broke critical applications. Despite the risks, patching remained a top priority. The mantra: Patch or perish.
Information Security and SearchSecurity.com took a firsthand look at the state of patching by visiting a number of enterprises on Patch Tuesday--the second Tuesday of every month when Microsoft releases its security updates. Microsoft shops are the easiest and most visible targets, but they also have the broadest array of tools and support. Our experience told us that we'd find a mix of disarray and mature processes and infrastructure. While laggards remain, it appears that enterprises have turned the corner on patching. (See Patch Tuesday)
In a recent survey, Information Security found that patching is falling on security manager's priority list. In terms of what's making their jobs harder, patching falls below user ignorance, lack of management support and securing remote connectivity. And security managers say they're able to effectively respond to threats and incidents.
This isn't to say that patching is passÉ. It seems that we've finally reached the point where patching isn't the painful fire drill it once was. Through a combination of good processes and tools, enterprises have developed systems that provide relatively quick remediation for vulnerable systems.
Each of the enterprises we visited on Patch Tuesday reported that managing the vulnerability lifecycle, not just the patch, is the key to plugging holes. Building repeatable processes, ensuring everyone knows their roles, and having plans for dealing with failed installations have smoothed out the patch process.
Aiding enterprises is the wide array of automated patching tools at their disposal. Give Microsoft credit; its Windows Update, SUS and WUS are essential for SMBs. Enterprise-class systems--such as those from PatchLink, Big Fix and Citadel--make testing, deployment and verification easier. Configuration management systems provide enterprises with the ability to lock down systems they can't patch. And quarantining techniques, such as VLANs and endpoint security systems, allow security managers to isolate unpatched machines until they can be fixed.
The quality of patches is also improving. Software vendors are doing a much better job of building and testing patches before releasing them. Likewise, the patch tool vendors are doing a much better job of supporting customers with the installation and troubleshooting of patches.
Are patching systems and processes perfect? Every system is fallible. Vulnerability scanners aren't reliable enough, and enterprises, according to vulnerability assessment vendor Qualys, are still leaving internal servers unpatched for as long as 60 days.
But, the apparent maturation of the process and systems is serving enterprises well. And, if enterprises remain dedicated, their processes will only improve with time, making patching just another part of the IT routine.