Despite (or perhaps because of) vendors' attempts to release patches on regular intervals, enterprises are still racing to seal holes in their infrastructures. Every extra minute a system remains unpatched is another opportunity for worms, backdoors, rooters and Trojans to infiltrate the network.
In many enterprises, patching is a messy, time-consuming process through which security teams must lab-test new code before welding it onto production machines. Every time you roll out a patch, look for obstacles and make adjustments that simplify the patch process to help you better plan patch deployments; save time and money; reduce errors and disruptions; and improve overall security. Inventorying and mapping your assets give you the ability to prioritize patches and their installations. Target high priority and high-value assets, then stage the remaining systems sequentially.
Inventorying assets is a continuous process and involves various security tools, including vulnerability scanners such as Nessus, Internet Security Systems' Internet Scanner and eEye Digital Security's Retina; mapping tools such as Nmap and p0f; and patch-level detection software. Some vendors, such as Tenable Network Security, are adding OS fingerprinting to their discovery tools, allowing for the detection of devices and information about what OSes and services they're running. Many automated patching systems include network mapping engines and service/OS discovery tools that list machine names, IP addresses, OS version, risk level, vulnerabilities and missing patches.
Welding shut security holes is as much about managing people as it is the patches. You need to develop lines of communications, cooperation agreements and processes for patch deployment. To security managers, it's a matter of utmost urgency, but to business managers -- those who own different pieces of the infrastructure -- it means lost production time as their servers are taken offline for reboots.
Security departments should set expectations and guidelines for patching across the enterprise. With historical data and a clear sense of ROI and the criticality of patching, you can create a system in which business units are required to install critical patches immediately; moderate patches during predefined maintenance cycles; and routine patches during periodic updates. Enlist business units to carry the burden of patching and make them accountable, enforcing requirements and maintaining standards through SLAs. You can use the information gleaned from successful, as well as botched, patch deployments to show each uncooperative department how its delay in deploying a critical Windows patch lead to a worm infection that cost significantly more downtime and remediation than the patch would have cost.
Deploying patches is only half of the patch management process, the other half is verification. Whether you're using patch management systems, automated tools like Windows Update or SUS, or manually installing patches, you always need to verify that the patch is installed correctly and is working properly.
Automated patch management tools are infamous for updating the registry keys to reflect full installation, yet failing to recognize that the patch download was disrupted. Then there are the patches that just don't work or undo the fixes installed by previous patches. If these issues aren't corrected, they will create problems that will slow future patching and performance. This is especially important if the security department isn't the one doing the patching. Network and business unit managers will swear up and down that they deployed a patch, but verification is up to you.
You'll need to pull out those same discovery, vulnerability assessment and penetration testing tools we talked about earlier. When used in concert, these tools will verify the effectiveness of your patch rollout, identify which systems need further work and measure the level of residual exposure. Each system that you fail to remediate in a given cycle is a threat to your enterprise.
Enterprises must develop their own metrics for measuring the success of their patch management programs. In the end, by employing these methods for continual improvement, your patch management system will reduce the vulnerability exposure and remediation time associated with enterprise-wide patch deployment.
About the author
George Wrenn, CISSP, ISSEP, is a technical editor for Information Security magazine and a security director at a financial services firm. He's also a graduate fellow at the Massachusetts Institute of Technology.