News Stay informed about the latest enterprise technology news and product updates.

Security researcher wins Microsoft Blue Hat prize for kBouncer

A new defensive technique that can be used in Intel-based processors could thwart memory-based return oriented programming attacks.

Microsoft has rewarded a researcher for his creation of a technique that can help defend against memory-based return oriented programming attacks.

We identified security defense talent that we may never have encountered otherwise, and helped the world get to know them too.

Katie Moussouris,
senior security strategist, Microsoft

Vasilis Pappas won the grand prize in Microsoft's Blue Hat contest, designed to get researchers to design memory-based defenses. Pappas earned $200,000 for kBouncer, a fully transparent return-oriented programming (ROP) mitigation technique. It's based on runtime detection of abnormal control transfers using hardware features found on Intel processors, according to Microsoft.

Pappas' method uses Intel processors' Last Branch Recording, or LBR, feature to detect ROP when system calls are made. It tracks the destination address of API return control transfers. The method is considered to be a practical and functional short-term mitigation technique that would work against most ROP attacks today, Microsoft said.

Two other winners received cash prizes for their research entries. Second-place winner Ivan Fratric received $50,000 for the creation of ROPGuard, a system that can detect and prevent the currently used forms of ROP attacks at runtime. Jared DeMott took the third-place, $10,000 prize for his entry, /ROP, a system that lowers the effect of address space disclosures and mitigates known ROP exploits.

The Blue Hat contest winners were announced at the Black Hat 2012 Briefings in Las Vegas. Microsoft announced its Blue Hat contest last year as a way to get security researchers to focus on defensive techniques to thwart memory-based attacks. The software giant does support a bug bounty program to reward researchers who discover vulnerabilities in Microsoft products.

More from Black Hat 2012

See more of's special coverage of Black Hat 2012.

The Blue Hat contest has helped connect Microsoft with talented researchers and create defensive techniques that have an immediate impact on the security industry, said Katie Moussouris, senior security strategist at the Microsoft Security Response Center. Features from Fratric's RopGuard submission have been integrated into the technical preview of the Enhanced Mitigation Experience Toolkit (EMET) 3.5. It's possible that other elements of the context submissions are incorporated into EMET or other products.

"One of the goals we set out to accomplish with this contest was to create both an incentive and an opportunity for fame and fortune in the area of security defensive research that never existed at this scale before," Moussouris wrote. "We identified security defense talent that we may never have encountered otherwise, and helped the world get to know them too."

Microsoft admits that techniques can be bypassed

Microsoft has acknowledged the difficulty of developing and implementing defensive security technologies. It shouldn't take long for cybercriminals to design an attack that would bypass Pappas' mitigation technique, said Matt Miller at Microsoft's Security Engineering Center.

"It is believed that attackers would be able to accomplish this in most cases with a low to moderate development cost," Miller wrote in the Security Research and Defense blog. "Specifically, imposing these checks on specific APIs (as in the prototype) may be prone to bypasses, and imposing checks only on returns does not mitigate all methods of chaining gadgets."

In fact, all three winning Blue Hat contest entries rely on techniques that are likely to be bypassed by cybercriminals, Miller said. The entries will have a short-term impact, but over time, cybercriminals will focus their resources on evading detection.

Miller called Fratric's research "novel," but said an attacker can adapt to the checks it employs at relatively low cost. ROPGuard limits checks to certain critical functions, but an attacker could bypass those functions by attempting to call a lower-level API, he said.

DeMott's technique could be "bypassed by leveraging gadgets that are in the set of valid return sites, or by using a gadget chaining method that does not involve a return instruction," Miller wrote. "The fact that this solution does not fully address all forms of code reuse limits the expected long-term impact of the design as described."

Dig Deeper on Emerging cyberattacks and threats

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.