Although independent security researchers discover more vulnerabilities than their vendor counterparts, the most critical vulnerabilities are discovered by vendor research organizations, according to a report issued Tuesday by IBM.
"You have to ask yourself whether you're providing more responsive patching and protection techniques, or are you doing it for self driven egocentric reasons."
Kris Lamb, Director, IBM's Internet Security Systems X-Force team
The report, issued by IBM's Internet Security Systems division, summarizes security statistics over the first half of 2008. It highlights the ISS X-Force research and development team's observations over the first half of the year and points out any new trends that researchers are tracking.
The report was critical of independent security researchers, drawing attention to statistics that showed independent researchers are almost twice as likely to have exploit code published on the same day as their vulnerability disclosure than vendor-driven research organizations. Over the past year and a half, independent researchers discovered 70% of all vulnerabilities that were not anonymously disclosed, but vendor research organizations found 80% of critical vulnerabilities, meaning those with a Common Vulnerability Scoring System (CVSS) base score of 10.
"You have to ask yourself whether you're providing more responsive patching and protection techniques, or are you doing it for self driven egocentric reasons," said Kris Lamb, director of IBM's X-Force research team.
There may be more proof-of-concept code available for issues discovered by independent researchers because they feel they have to provide more proof of the validity of a vulnerability, Lamb said.
"At what point are they providing too much information that doesn't help the community respond appropriately?" he asked. "There is a very fine line around the right amount of information and too much information."
Lamb pointed to the DNS cache poisoning flaw discovered by Dan Kaminsky, director of penetration testing at IOActive Inc., as an example of where security researcher's egos can put people at risk. Kaminsky organized a massive multivendor patch release in early July, and initially wouldn't share any details of the vulnerability with security researchers. What followed was a tidal wave of speculation, which resulted in reverse engineer Halvar Flake, correctly guessing the details of the flaw.
"Despite Dan Kaminsky's doing what he thought was right, partial disclosure failed once the information got outside of his control," Lamb said. "I don't think at that point it was about protecting the Internet, it became about personalities and egos."
"It created a situation where eventually the security community put customers more at risk by the speculation and flooding of more information, which followed with exploitation code," Lamb said.
Internet Security Systems Inc. (ISS) has waded into this debate on a number of occasions in the past, most famously at the Black Hat conference in 2005 when one of the company's researchers, Michael Lynn, quit because ISS and Cisco Systems Inc. were pressuring him not to reveal details of a flaw he had discovered in Cisco's IOS software. The flaw was patched by the time of his talk, and Lynn quit his job rather than agree not to talk about it at the conference. Black Hat organizers went so far as to tear the pages containing Lynn's presentation out of the conference proceedings book.
A number of security researchers took issue with ISS's conclusions about vulnerability research and disclosure, saying that, especially in the case of the DNS vulnerability, there was little else that could have been done to help protect users.
" What do you do when you figure out how to break the Internet so completely that it stops working? How do you make people take you seriously without letting the cat out of the bag?" asked Billy Hoffman, manager of HP Web Security Research Group at HP Software Inc. "It's a very difficult situation. I don't think that it was about Dan's ego at all. The very fact that he approached Paul Vixie and worked with him shows that. And when Dan started realizing people weren't taking it seriously, he didn't fall back on the attitude of, I'm Dan Kaminsky, listen to me. He went to Tom Ptacek and others and had them talk about how serious it was. The way he handled it is as well as he could have."
Nate Lawson, principal at Oakland, Calif.-based Root Labs, said the only metric that matters is patch acceptance. Lawson is the co-designer of the copy protection scheme for Blu-Ray discs, and has conducted independent security research on cryptographic algorithms, hardware and software.
"What is the patch rate of announced vulnerabilities compared to the rate when or if an exploit appears?" Lawson asked. "The ISS study ignores the critical question of how both organizations and independent researchers can give companies the info or tools necessary to increase their patch rate."
Nitesh Dhanjani, a leader of application security services and independent security researcher, agreed that "releasing exploit code churns the security media circus and gets people notoriety.
"If you are well-known, people have higher expectations of you, leading to philosophical and ethical debates on the matter, and this kicks in the law of diminishing returns against disclosing exploit code on day one," Dhanjani said. "You have the lesser known researchers on one end, followed by the more well-known researchers, and corporate businesses on the other end. If you look at this train from left to right, the incentive to publish exploit code on day one decreases, and may in fact work against you and against the expectations people have of you."
Lamb said it could be time for new standard for vulnerability disclosure that addresses the differences between an independent and a vendor-driven researcher.
"When you're talking about responsible disclosure that has muscle of a vendor behind it, there tends to be a difference in standards," Lamb said.
But Hoffman and others see it differently. "I think this actually revalidates the need for third party researchers," Hoffman said. "There was a very adversarial relationship with the security community back in the 1990s, then things started to change. Microsoft and the MSRC did a great job changing the way they handled things. Recently we've seen a contraction of that. We have people offering to pay for zero-days, and it seems like vulnerability research has taken a couple steps back recently. What Dan has done has given a lot of credit back to the community. If he wanted to just break things, he would have sold that exploit, and he didn't."
Executive Editor Dennis Fisher contributed to this report