Information Security

Defending the digital infrastructure

igor - Fotolia

Manage Learn to apply best practices and optimize your operations.

Bug bounty programs narrow the crowd

Data shows that more companies are moving away from crowdsourcing and adopting invitation-only awards programs. Do higher-quality submissions result?

Enterprise-sponsored bug bounties continue to grow in popularity. With that growth have come debates about the best ways to manage the risks involved with paying unknown researchers and hackers to find vulnerabilities. 

Some companies have adopted invitation-only bug bounties, leading to a higher percentage of quality submissions than vulnerabilities identified in public crowdsourcing programs. Other enterprises, especially those in the technology field, are ramping up their bug bounty programs and offering community researchers a clear path for reporting vulnerabilities and potential fixes.

Jason Shirk, senior director of the Microsoft Security Response Center, blogged about a number of changes to the software giant's bug bounty programs last August. With a heavy focus on the newly released Windows 10 operating system, the most notable change is the reward for a qualifying "defense idea," known as the Bounty for Defense submission. The maximum payment doubled from $50,000 to $100,000, which Shirk says will "bring defense up on par with offense."

Under the Microsoft Bounty Programs, the Bounty for Defense must accompany a Mitigation Bypass submission, described as "novel exploit techniques against the latest operating system." In 2013, James Forshaw was awarded $100,000 for an exploit against Windows 8.1 using unsafe COM objects. Microsoft pays up to $100,000 for a qualifying Mitigation Bypass submission, bringing the maximum payout to security researchers to $200,000, although the company has run contests in which it has paid more.

Facebook is also investing more in its security efforts, bringing former Yahoo CISO, Alex Stamos on board as its chief security officer in June of last year. During Stamos' tenure at Yahoo, he and his team of "Yahoo Paranoids" established a formal bug bounty program, which has paid out more than $1 million in rewards. One can only expect Stamos to continue the tradition of bug bounties at Facebook, whose checkered bounty program has paid out comparable amounts.

To date, there is no cap on Facebook's bug bounties: Individual researchers have earned more than $100,000 for multiple valid submissions, according to the company. The minimum payment is $500. In addition to professional researchers, the youngest bounty hunter on record is 13 years old. Like other bounty programs, the awards are made at the discretion of Facebook, and a few recipients have been hired by the company to work on its security team.

Invitation-only programs

Bugcrowd -- providers of a fully managed platform that hosts "private" and crowdsourced bounty programs for numerous companies, including Dropbox, Fitbit,, Pinterest and Western Union -- took advantage of the researchers gathered at last summer's Black Hat USA 2015 to release its inaugural The State of Bug Bounty report. The 21-page report, which shows that incentivized, invitation-only programs are the best way to improve the signal-to-noise ratio for a company's security assessment program, appeared to garner a lot of attention during the conference.

The following are a few highlights from the report:

  • During the two-year reported period, 17,994 unique researchers from 147 countries collectively submitted 37,227 bugs, of which 566 people were rewarded a total of $724,014. The top reward was $10,000 for a single valid submission.
  • The number of invitation-only programs running on the Bugcrowd platform recently surpassed public programs. Invitation-only programs offer better signal-to-noise ratio results: 36% invitation-only submissions are marked valid compared to half as many (18%) from public programs.
  • Companies running bug bounty programs typically include large tech companies, though Bugcrowd's platform data shows that a growing number of organizations outside the high-tech industry are joining the bug bounty party.

Understanding these and other metrics can provide enterprises with a valuable security playbook, such as the type and severity of vulnerabilities found in real-world production sites. According to Bugcrowd's research, on average, 4.39 "high- to-critical vulnerabilities" were identified per program. The most common vulnerability is cross-site scripting, representing close to 20% of the bugs found. Researchers on average were paid for one in five submissions.

Bad Bugs: The Priority Matrix

Top researchers

One of the most successful and well-established researchers at the moment is Bitquark. While an exact figure wasn't disclosed, Bitquark has earned tens of thousands of dollars participating in Facebook, Google and other enterprise bug bounty programs, including some that Bugcrowd manage. That amount is substantial compared to other researchers who, according to The State of Bug Bounty findings, earned an average of $1,279.18 annually based on 6.41 submissions.

Some security researchers are interested in payouts, others are focused on ranking points, and the remainder is fine with a combination of the two, according to Jason Haddix, the well-known hacker who topped the Bugcrowd leaderboard in 2014 before he joined the company as its director of technical operations in May. His team performs triage and validates vulnerabilities from about 16,000 community researchers, effectively functioning as the conduit between the hunters and the companies that are running the bounties on Bugcrowd's platform. Their data showed that about 50% of researchers are from India (31%) and the United States (18%).

Some of the researchers who were gathered at Black Hat and attending Microsoft and Bugcrowd functions, admitted they were definitely all about the money -- points and notoriety didn't matter to them. Others saw the value in the points, as a way to gain a competitive edge over fellow bounty hunters; higher rankings come in handy when the invites get sent out for the invitation-only programs. This group tended to be younger, less established and more in need of recognition.

Bugcrowd provides other avenues for lesser-known researchers to get their name out in the security community: guest blogs, interviews and podcasts are all popular brand-building vehicles for security researchers.

Successful hunting can also lead to employment. Bitquark was hired as a security researcher for Telsa Motors after reporting a valid SQL injection flaw to the electric car manufacturer under its bug bounty program in 2014.

Hacking into computerized vehicles is a popular target. Kevin Mahaffey, co-founder and CTO of security firm Lookout, and Marc Rogers, principal security researcher at CloudFlare, presented security research on vulnerabilities in Tesla's Model S at DEF CON and Black Hat USA 2015. About the same time, Telsa Motors hired Chris Evans, the former head of Google's Project Zero team of hackers tasked with finding zero-day vulnerabilities in Google-related products, as its head of security.

In 2015, Telsa started a bug bounty program for its public-facing Web applications via Bugcrowd. According to Bitquark, who manages the effort, the platform is ready to use, and security researchers are well supported and can participate in multiple bug bounty programs. Earlier this year, General Motors launched its own vulnerability disclosure program via Hackerone, albeit without an actual payout.

Happy hunting

With his role at Tesla keeping him busy, Bitquark isn't participating in as many bug bounty programs, though he still enjoys applying his skills to stimulating projects when he can find the time. His strategy is to focus on the areas that will have the highest impact and produce the best results for the company, particularly when the vulnerabilities he identifies are fixed. These are often logic bugs.

"It isn't something I purposefully sit down and say ‘OK, now I will do one, two, three,'" he says. "It's more second nature to me than that."

Bitquark approaches an app as if he were looking at it from inside the code. He identifies the steps involved in a process and messes with them, skipping a step, repeating a step, even breaking or ending the process before it is supposed to finish. He also looks at validation bugs. 

"I definitely keep in touch with other researchers to stay on top of the latest in research techniques and findings," he says. "In some cases, I even discuss a roadblock with another researcher as a means to break through it. This isn't the case very often, though."

Sloppy code

The consensus amongst white hat security researchers is that they want to do the right thing. Most of them say they try to help companies find and fix vulnerabilities in their software and hardware -- and they enjoy the challenge in doing so. Although some researchers admit that if an organization won't pay for an exploit, they may attempt to sell it elsewhere.

Despite the risks, community researchers can help organizations better secure products and improve their internal testing processes. Most vulnerability submissions in private and public bug bounty programs uncover coding errors and inadequate testing during the development lifecycle (in some cases, due to time pressures and a lack of resources). Third-party integration comes with its own challenges. Many bug bounty programs, such as Facebook's, are to acquired products and services. 

"Every bug found is important," says Haddix. "It's easy as a researcher to get frustrated with the developers when you find a no-brainer bug -- they should know better." Some companies are paying (sometimes big money) to find a bug that should not be left for a big bounty researcher to find.

"If the money is good, we don't mind," agrees one researcher, who requested anonymity. "If it's for points, and we don't care about the points, then spending time finding a no-brainer bug can be viewed as a waste of time." 

Article 2 of 5

Next Steps

Microsoft beefs up cloud security with bug bounties

What bug bounty programs can do for the enterprise

Bug bounty programs at non-tech companies

This was last published in March 2016

Dig Deeper on Penetration testing, ethical hacking and vulnerability assessments

Get More Information Security

Access to all of our back issues View All