Pitching patch: RFP bakeoff

SearchSecurity.com

Pitching patch: RFP bakeoff

Patch management is a never-ending challenge. Organizations ranging from the 60-seat shop with a three-person IT staff to international Fortune 1000 companies balance the cost and resource drain of prompt, diligent patching against the risk of exposing important assets to exploits that appear with alarming speed.

Inadequately tested patches can break systems; VA scanners are intrusive and not always accurate; patches are interrupted or fail for a variety of reasons, requiring painstaking validation and additional remediation; and the growing army of mobile users connected intermittently to the network get "missed," posing an uncontrolled threat to the enterprise.

Against this backdrop, Information Security challenged automated patch management vendors to respond to a request for proposal (RFP) from a hypothetical mid-sized company with very real problems: an overtaxed IT staff coping with a highly distributed environment, and lagging patch deployments and consequent successful malware attacks.

Our Methodology

Information Security invited more than 20 patch management vendors to respond to a request for proposal (RFP) for a hypothetical company of some 2,500 employees spread among many offices. We chose the seven best proposals based on the following criteria:

Clear and thorough description of the product technology and methodology.
On-point responses in the context of the scenario's requirements, system

environment and patch-related problems.
Specific deployment recommendations.
Quality and professionalism of presentation.

We submitted seven proposals for evaluation by our panel of experts and followed up with the vendors to clarify certain points. The vendors' responses were reviewed to resolve any outstanding issues.

Final assessments were based solely on the proposals. Prior knowledge of the company's products or track record wasn't considered. Responses, not reputations, were evaluated.

We selected the seven vendors who did the best job presenting comprehensive solutions tailored to our scenario: BigFix, Citadel Security Software, Configuresoft, Everdream, PatchLink, St. Bernard Software and Shavlik Technologies. We then asked a panel of four infosecurity experts to analyze and report on the proposals.

What we found is RFP responses can tell you a lot about the vendor you're dealing with. Click the links below to read the summaries of the RFP responses.

View the results chart here.

A revealing exercise
Overall, we were disappointed in the responses to our RFP. Most of the proposals read like stock replies we would get in brochures and product description sheets in response to filling out online forms.

However, asking vendors to put their best foot forward and describe how their technology might work in real-world scenarios revealed strengths and weaknesses in different ways than we might have seen in a lab comparison.

In most cases, the vendors came up short in explaining their technologies and in the quality of their responses, and these seven were deemed the best among more than 20 submissions. BigFix's and PatchLink's proposals came closest to what we'd expect to see as a potential purchaser.

We wouldn't venture recommendations based on this process, but it was informative to consider the different technologies and approaches: managed service, use of third-party scanners, agent-only solutions and mixed offerings. Each has its strengths and gives potential customers much to consider before deciding how best to ease their patch management burdens.

Read Jon Oltsik's Demand good proposals to learn how to improve prospects for RFPs that actually respond directly to your requirements.

MEET THE PANELISTS

TOM BOWERS, CISSP, PMP, CEH, is a technical editor for Information Security and a manager of security operations at a pharmaceutical company.

JAMES C. FOSTER is a technical editor for Information Security and deputy director of global security solutions development at Computer Sciences Corp.

PETE LINDSTROM, CISSP, is research director at Spire Security and a contributing editor for Information Security.

JON OLTSIK is a senior analyst at the Enterprise Strategy Group, and previously VP of marketing and strategy at GiantLoop Network and senior analyst at Forrester Research.

About the author
Neil Roiter is Information Security magazine's senior technology editor. Send your thoughts on this article to feedback@infosecuritymag.com.

27 May 2005