The initial cost of purchasing an application is only a small part of the total cost of ownership. Software security can have a big impact through difficult-to-assess costs such as patch management, repair downtime and liability issues if a computer or network is compromised.
"These are costs of owning the software, and we can calculate them in hindsight, but how do we measure, put bounds on them or reduce them through our purchasing decisions?" asked Herbert Thompson, an adjunct professor at the Florida Institute of Technology's computer science department. "We have to evaluate this risk; we need some guidance on how to predict these possibilities."
In the past, security practitioners have relied heavily on reviews, marketing literature and performance reports that include vulnerabilities and an application's susceptibility to virus and worm attacks. Thompson called these methods "really dangerous" because marketing materials are biased and reviews provide little information about deployed security. Performance reports are little better because counting vulnerabilities or viruses only shows which platforms or products more people are attacking, measuring what Thompson called "a weird hybrid of security and threat profiles." Also, counting doesn't consider things like severity or what the vendor is doing to make sure that those types of things don't happen again.
Thompson suggested several alternatives to evaluate products.
Asking the hard questions
When meeting with software vendors ask proactive questions about security flaws to determine any potential issues before buying a product:
- Do you have a dedicated team to assess and respond to security vulnerability reports in your products?
- What is your vulnerability response process?
- What process improvements have you made as a result of vulnerabilities reported in your software?
- What is your patch release strategy?
- What training do your development and testing groups receive on security?
- What percentage of your test team is focused on security?
- Does your company monitor the latest attack trends in the underground community and consider how those trends may affect your software?
- Do you patch all currently supported and vulnerable versions of your applications/platforms at the same time?
- What are the terms and period of your security support agreement?
Using Red Teams: Testing internally
"If you don't internally test your applications, someone else will," Thompson noted and recommended developing a small, focused test team to attack the product like an intruder would. Using such a team will provide an independent feel for the security of vendor applications and the advantage of knowing how your company would deploy, use and configure the software. "Red Team" is a government and industry term for a focused group of security testers that attack an application or system to test it.
Independent security assessments can be effective in providing a comparative analysis and acceptance testing, and can be inexpensive when compared to the total cost of ownership of deploying potentially vulnerable software. Many vendors offer assessments -- Ernst & Young, TruSecure, IBM, Foundstone, @stake and Security Innovation among them.
Developing a security-aware organization begins, in part, with IT personnel understanding how the software will be used. Instead of focusing on deploying add-on protections for security, such as firewalls and antivirus, Thompson said IT needs to begin mitigating risk through purchasing and deployment decisions too.
"Demand security from your vendors," concluded Thompson. If you don't motivate the vendors by asking security questions, nothing is ever going to change.