Co-authored by Sammy Migues and Jacob West
By now, you should have heard about the Building Security In Maturity Model (BSIMM) project, especially if you are a software security person. (No? Then read this.) Maybe you've even downloaded a copy of your own to peruse (it's free under the Creative Commons license).
Either way, it's time to get yourself a new copy, because BSIMM-V has just been released. Remember, because BSIMM is completely data driven, the BSIMM-V document is different than what you may have read in the past. That's how science goes.
In this short piece, we're going to focus on BSIMM-V facts and figures. The numbers are about real software security initiatives doing real work to secure the software that you use every day. This is no ephemeral top ten list from the bug parade. This is a set of facts about the real state of commercial software security on planet Earth.
Who is the BSIMM community anyway?
The BSIMM project is spearheaded by three co-authors (the same three who wrote this piece you're reading now). We are directly involved in gathering data in person from each of the BSIMM firms. The data we gather directly through observation describes the work of 67 software security initiatives, from firms including: Adobe, Aetna, Bank of America, Box, Capital One, Comerica Bank, EMC, Epsilon, F-Secure, Fannie Mae, Fidelity, Goldman Sachs, HSBC, Intel, Intuit, JPMorgan Chase & Co., Lender Processing Services Inc., Marks and Spencer, Mashery, McAfee, McKesson, Microsoft, NetSuite, Neustar, Nokia, Nokia Siemens Networks, PayPal, Pearson Learning Technologies, QUALCOMM, Rackspace, Salesforce, Sallie Mae, SAP, Sony Mobile, Standard Life, SWIFT, Symantec, Telecom Italia, Thomson Reuters, TomTom, Vanguard, Visa, VMware, Wells Fargo, and Zynga.
By the way, we added a data freshness constraint to the model with BSIMM-V. We now exclude measurements older than 48 months. This requirement caused five firms to be removed from BSIMM-V. As the data set ages, we intend to decrease the freshness window to 36 months to better align with business cycles.
What is the BSIMM?
The BSIMM is a measuring stick for software security. The best way to use the BSIMM is to compare and contrast your own initiative with the data about what other organizations are doing contained in the model. You can then identify goals and objectives of your own and look to the BSIMM to determine which further activities make sense for you.
The BSIMM is not a software security methodology. To make this clear, consider that the BSIMM can be used to measure Microsoft's SDL, but it is by no means a replacement for the Microsoft SDL.
BSIMM by the numbers
Table 1 shows how the BSIMM Project has grown over the years.
As you can see, at this stage of the game, the BSIMM describes the work of 2,890 full-time software security professionals who are attempting to help 272,358 developers build more secure software. They have some help from the "satellite," which is made up of developers, architects and people in the organization directly engaged in and promoting software security, but not as full-time software security group (SSG) members.
The most important number in the Table 2 below is the "SSG average of averages," which is the number of SSG members on average each firm has per 100 developers. Ever wonder how big your firm's SSG should be? We wonder also, but we do know how big the SSGs are at 67 firms.
There are some other interesting facts shown below as well. Like the fact that software security initiatives are ongoing and not a fire-and-forget exercise.
Table 3 below shows just how many firms make use of the 111 activities in the BSIMM. Each activity has a label (like SM1.1) and is described in detail in the BSIMM document. See, it turns out we do know how to do software security! We even know who is doing what. Now what we need to do is spread adoption of software security to all firms creating software. You can help.
How does your firm compare?
Here's what happens when you measure a new firm using the BSIMM measuring stick. You can directly compare how your software security initiative stacks up against the other 67 firms in BSIMM-V.
Is your firm a financial services institution? Well, we can compare you to 26 other financial services firms. Are you an ISV? We can compare you directly to 25 other ISVs. Measurement is a powerful tool that drives both budgets and improvement.
Nobody wants to be the slowest zebra in the zebra pack. Is your firm the slowest zebra? (See Table 4.)
We have created a spider diagram (Figure 1) as a way of visualizing a low-resolution comparison based on 12 practices. The 111 activities in the model fit directly into the 12 practices.
Our spider-graph-yielding "high-water mark" approach (based on the three levels per practice) is sufficient to get a low-resolution feel for maturity, especially when working with data from a particular vertical or geography.
One meaningful comparison is to chart your own firm's maturity high-water mark against the averages we have published to see how your initiative compares.
The BSIMM community
The 67 firms participating in BSIMM-V make up the BSIMM community. A moderated private mailing list with over 200 members allows SSG leaders participating in the BSIMM to discuss solutions with others who face the same issues, discuss strategy with someone who has already addressed an issue, seek out mentors from those further along a career path and band together to solve hard problems.
The BSIMM community also hosts annual private conferences where up to three representatives from each firm gather together in an off-the-record forum to discuss software security initiatives. In fall 2012, 28 of 51 firms participated in the third annual BSIMM Community Conference in Galloway, N.J. In spring 2013, 10 of 15 firms with a presence in the E.U. participated in the Second Annual BSIMM Europe Community Conference in London.
The BSIMM website includes a credentialed BSIMM community section where information from the conferences, working groups and mailing-list-initiated studies are posted.
Would you like your firm to be included in the BSIMM community? Give us a shout. BSIMM-V is the latest snapshot of a growing and evolving set of real data about software security. The more data we have, the better off we all are. It's science time.