Testing security patches
I am looking for best practices on how to test security patches before applying them. After what kind of testing do other shops consider a patch safe? I know it has to vary for different environments, but when does one consider that enough testing is enough testing?

In a perfect world (and I am dreaming "out loud" here), I would use an automated tool that would drive lots of different tests against a patched server and overnight it would give me a report stating that the results matched some predefined criteria allowing me to decide if the patch is safe, i.e. does not create other problems. Does such a tool exist?


    Requires Free Membership to View

    Login

    SearchSecurity.com members gain immediate and unlimited access to breaking industry news, virus alerts, new hacker threats, highly focused security newsletters, and more -- all at no cost. Join me on SearchSecurity.com today!

    Michael S. Mimoso, Editorial Director

    By submitting your registration information to SearchSecurity.com you agree to receive email communications from TechTarget and TechTarget partners. We encourage you to read our Privacy Policy which contains important disclosures about how we collect and use your registration and other information. If you reside outside of the United States, by submitting this registration information you consent to having your personal data transferred to and processed in the United States. Your use of SearchSecurity.com is governed by our Terms of Use. You may contact us at webmaster@TechTarget.com.

Unfortunately, recent history makes it so that the best practice about patches is to install it as soon as possible, and the IS staffs are negligent if they delay. If you look at security problems having a window of vulnerability, delaying the install of a patch merely makes you more vulnerable. The problems they fix are so frightening that delay is bad.

You are right, in a perfect world, there would be a set of tests you can run against a server or workstation to make sure that things work properly. However, we don't get told what's in those patches, and so it is hard to know what to test. Often, the description of the problem is intentionally vague so as not to make it utterly obvious how to write an exploit program that can compromise machines during the time that IS groups are rolling out the fix.

In the case where a security fix is a browser fix, you are balancing the browser not working against one of your 10,000 users reading the wrong Web page that infects their machine. Pick your poison.

There aren't automated tools that can tell you if the patch is safe. It would be nice if there were. But alas, vendor software is not very good, and there's a reason they're creating the patch. Some Microsoft people I know told me that it costs them US$100,000 to release a patch. They don't do it because they want to. They do it because the problem is embarrassing enough to spend $100K to deliver a fix.

For more information on this topic, visit these other searchSecurity resources:
News & Analysis: Keeping up with patchwork near impossible
Tech Tip: Managing the patchwork mess


This was first published in February 2002