I have to admit that I was amused when my editor asked me to comment on an e-mail she received. Apparently WatchGuard, or hopefully it was their PR firm, had a survey performed at RSA last month. I thought that the title of the e-mail message was also worthy of a sarcastic comment or two, "Security Industry Wants MORE Government Action." And yes, the "more" was in all capital letters.
When I see press releases about surveys, my academic background kicks in. My undergraduate degree is in psychology, and I also took extensive graduate courses in surveys and statistics. I even performed several surveys that were used as evidence in criminal cases. As I hoped to put that phase of my life behind me when I started doing computer work, I just shake my head ruefully as I see study after study come out on people's opinions about security.
By this point in time, according to all the opinion surveys of "security experts," analyst firms and the security industry, we should have experienced several dozen electronic Pearl Harbors, and the Internet should have been destroyed as we knew it. We should only be so lucky. So much for the opinion of all of us security experts. I only hope our bosses have a bad memory so they can actually trust our opinions.
Now, when I saw the WatchGuard study, I thought I would look at it a little more carefully. What qualified people as representing the security industry? How was the sample collected? How
You have to ask if the presence of a person at RSA qualified that person as representing the security industry as the survey seemed to indicate. Besides people in the security profession, I saw other RSA attendees who just wanted to learn. There were also reporters, security guards and even a few people cleaning the bathrooms. I'm sure they would happily fill out a survey for a free T-shirt.
On top of this, the answers were truly all over the place, demonstrating that there was little agreement on anything. According to the press release, only 17% of people believe that government regulations are effective, yet the survey data also stated that 68% believe that there should be more regulations. Back in the old days when I had to take statistics, we would call this a validation question, and the validation would have appeared to fail. I guess fundamental mathematical principles have changed in the last 10 years.
Opinions are like noses; everybody has one. And that is the publishable version of the quote. Even if the survey was otherwise statistically accurate, who really cares about what people think? Industry lobbying groups have infinitely more impact than a random survey of people when it comes to government regulations.
I can't really blame WatchGuard or its PR firm for conducting yet another questionable study that may garner media attention. Sadly, I'm sure that there are many reporters, editors, and Web sites that are dying for content and will be happy to cover this apparently poorly crafted survey over other boring content. As I previously mentioned, there has been questionable study after questionable study -- that time has proven to be outright wrong -- reported by the media. I guess WatchGuard and its PR firm are somehow convinced that putting out yet another poor study and offering an "expert" from WatchGuard is somehow more newsworthy than that its product recently won an industry award.
What have we come to when an organization like WatchGuard believes that instead of spending its money touting the need for and benefits of its products, it gives us information of dubious value? This isn't a commentary on WatchGuard, but rather a sad comment on the security industry media that will probably cover an apparently statistically invalid sampling of random "noses."
About the author
Ira Winkler, CISSP, CISM, has almost 20 years of experience in the intelligence and security fields and has consulted to many of the largest corporations in the world. He is also author of the forthcoming book, Spies Among Us.