News Stay informed about the latest enterprise technology news and product updates.

Harvard report: Metadata means there is no 'going dark' for the FBI

A new report from Harvard said data 'going dark' in the face of strong encryption shouldn't be a problem for law enforcement and intelligence agencies.

Since 2014, FBI officials, including director James Comey, have frequently claimed that data is "going dark" because...

of the increased use of encryption on digital communications, but a new report published by Harvard's Berkman Center for Internet and Society casts doubt on those assertions.

The FBI's stance has been that strong, end-to-end encryption and encryption of smartphones makes it more difficult for intelligence agencies to run surveillance operations on terrorists and criminals. The FBI has claimed it doesn't want Congress to legislate backdoors to circumvent encryption, but would rather companies willingly find a way to comply with legal requests for data. However, senators have begun working on drafting a bill to require access to encrypted data through "special access," which critics called mandatory backdoors. And both New York and California have seen bills proposed to ban the sale of fully encrypted smartphones.

In Don't Panic, the report authored by "a diverse group of security and policy experts from academia, civil society and the U.S. intelligence community," they not only agreed that weakening encryption offers more risk of harming innocent users than it does in catching terrorists, but they went beyond that argument to note that "going dark" may not be a real fear, because there will be other avenues for intelligence agencies to gain data.

The report authors said the issue may not be as bad as advertised for a number of reasons. First, the experts don't believe end-to-end encryption will be "adopted ubiquitously by companies, because the majority of businesses that provide communications services rely on access to user data for revenue streams and product functionality." Additionally, encryption ubiquity will be hampered by software fragmentation, and the lack of coordination and standardization between digital services right now.

The report authors also wrote that the Internet of Things (IoT) "has the potential to drastically change surveillance." Their theory is that IoT is expected to become widespread, and those devices could enable real-time capture of images, audio or video. "Thus, an inability to monitor an encrypted channel could be mitigated by the ability to monitor from afar a person through a different channel."

Jeff Schilling, chief security officer for Armor, based in Richardson, Texas, said it is still too early to predict this, because many people still don't have a solid definition for IoT and it is still possible that previous cybersecurity lessons can keep IoT secure.

"If we transition correctly between our current network-centric model, created by the advent of TCP/IP, to a data-centric model or Internet of Things, I think there are lots of opportunities to make surveillance harder for both the good and bad guys," Schilling said. "At the end of the day, just about everything eventually ends up as a database -- email, social media sites. If you can clearly identify who has access to it and encrypt it at rest, then that is a pretty hard bar to get over for someone to exploit."

The Harvard report also noted that even with fully encrypted communications, there is a lot of metadata available that can be used by law enforcement for surveillance. The reasoning stated is that metadata "needs to stay unencrypted in order for the systems to operate: location data from cell phones and other devices, telephone calling records, header information in email and so on."

This means that the "going dark" argument is somewhat overblown, because the report said this metadata "information provides an enormous amount of surveillance data that was unavailable before these systems became widespread."

Rebecca Herold, CEO of Privacy Professor, said "lawmakers and politicians have seemed to have a deaf ear to these facts." Though, she did note that former NSA Director General Michael Hayden famously said about the agency, "We kill people based on metadata."

Herold said that this metadata can often be more valuable than the encrypted messages being sent.

"Keep in mind, messages crooks and terrorists send are often not explicitly detailing what they are doing; they often are using code words," Herold said. "So, yes, the metadata is very valuable and often provides more insights than the coded message itself."

Herold said that the claims by FBI Director James Comey about "going dark" might be due to his not understanding how encryption works.

"It might be that they want to see everything involved and are not interested in the value of the metadata that they can access," Herold said. "It is like receiving a wrapped gift; the recipient looks at it and often thinks of all the wonderful things it is holding inside. But then, when it is unwrapped and the actual gift is revealed, there is no longer mystery, and often there is disappointment."

Next Steps

Learn why experts say lawmakers don't understand encryption backdoors.

Learn how metadata management relates to data governance.

Learn about weighing public safety costs against the benefits of end-to-end encryption.

Dig Deeper on Government information security management

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Where do you stand on the debate over encryption and intelligence gathering?
I don't have an issue with either. The government can tap our phone lines if they get a court order. That means there must be a reasonable reason for the surveillance request.  They are not just going to spy on everyone. That would be way to much data to sift through to find what they may be looking for. Encryption is not an end all for keeping secrets. It may be a hurdle for the government but they have more resources for cracking the code than the average user.

Ed Baldwin; This is really a discussion about the value of the analysis of Metadata, also known as Traffic Analysis or the analysis of elements external to the encrypted data payload vs. Crypto Analysis or the analysis or exploitation of the encrypted message.

While there is certainly value in Crypto Analysis, I tend to agree with Ms. Herold's position that there is much to be gained from Traffic Analysis.  If set up correctly, this should provide law enforcement the tools they need to find the bad buys while protecting the privacy of our citizens.

Last, though there are niceties to be maintained with our allies, I've no issue what so ever, with going after back-doors and exploiting encryption systems of our adversaries.  This position means that I'm acknowledging the need for LE to have the ability to perform Crypto Analysis.  Therefore, to protect our citizens and allies, the trick here is legislative.  When writing the law and more importantly the follow-on regulations, there must be real checks-balances rather than a rubber stamp from a FISA judge.  Perhaps this means setting up an IG that is not accountable to DoD or the Intelligence Community, but reports directly to the congressional (House and Senate) intelligence committees.

Thoughts on this approach?


the only people that should mind our phones being tapped are people doing things they shouldn't do.    This article has the undertone of "Don't check on me. No matter what I do,".

The other thing I noticed is everything in it was totally subjective and bent on "some possible thing that might appear later" that might possibly mitigate the actual problem which is TRUE exactly as stated by the FBI if phones were to be encrypted end to end.

I kind of agree. The ones that yell and scream the most about change are sometimes the ones abusing it. They do not want you to fix something they have been exploiting. This forces them to do more work to continue what they have been doing now that a light has been shown on the matter.

@Ed, I think we’ve seen that due to our legislators inability to work together that they would rather create and pass knee-jerk legislation than take a responsible path subject to checks and balances and common sense.
Nice article. I do think this is a touchy subject for a lot. Some feel the government is snooping into our lives more than it should. How do we know they do not have methods in place now? It's more of those who want to remain hidden that make the most noise. Like I have always said if you want to keep things private do not use the internet.. Plain and simple. What gets put out there stays there and there are plenty of people that want to snoop or save what they find.
Well, we shouldn't discount the fact that the least well-informed on this matter...those in the US Congress, will offer up the worst possible solution in the form of legislation. Even now some state legislatures are attempting to ban the sale of smartphones that encrypt data. Outside of the incumbent telco providers, who have historically sold out the privacy rights of Americans, none of the major corporations that make smartphones or handle/store customer data are interested in creating any means by which government can access or break the encryption deployed in their devices or services. Thanks to Mr. Snowden we have already been shown how treacherous and untrustworthy the federal government and its agencies can be when it comes to spying and illegally collecting data on all Americans using mass surveillance of their communications.
It’s not so much that they are snooping into our lives more than they should that bothers me. It’s more the sense of entitlement that I get from the agencies that want the information.