Lance Bellers - Fotolia

U.S. intelligence agencies cut off from Twitter firehose

Twitter ordered its business partner Dataminr to cut off the Twitter firehose feed access for U.S. intelligence agencies, but experts expect the NSA won't miss much.

Twitter has asked its business partner Dataminr to stop providing U.S. intelligence agencies with access to the Twitter firehose feed because the social media giant doesn't want to be associated with government surveillance.

Dataminr has access to the firehose -- the full stream of all tweet activity -- but Twitter, which owns a 5% stake in Dataminr, ordered that the National Security Agency (NSA), CIA and other government intelligence agencies be cut off from that data.

Twitter released a statement asserting this was nothing new and Dataminr shouldn't have allowed access in the first place.

"Dataminr uses public tweets to sell breaking news alerts to media organizations, such as Dow Jones, and government agencies, such as the World Health Organization, for nonsurveillance purposes," Twitter said. "We have never authorized Dataminr or any third party to sell data to a government or intelligence agency for surveillance purposes. This is a longstanding Twitter policy, not a new development."

John Inglis, a former deputy director of the NSA, told The Wall Street Journal this move was indicative of the troubled state of cooperation between the private sector and the government.

At the time of this publication, none of the parties involved -- Twitter, Dataminr, the NSA or the Department of Homeland Security -- had responded to requests for more detailed answers on the event.

Rebecca Herold, CEO of Privacy Professor, said the move might have been related to personally identifiable information (PII) in the Twitter firehose or to international laws.

"Twitter Terms of Service indicate that they do 'not disclose personally identifying information to third parties except in accordance with our privacy policy.' The type of big data analysis that Dataminr provided may have resulted in such revelations of PII, and so when this data was shared with others, [it] could have violated their own privacy promise," Herold said. "Also, the Twitter rules indicate that Twitter cannot be used 'for any unlawful purposes or in furtherance of illegal activities.' With new privacy laws worldwide, and also the new General Data Privacy Regulation in the EU, they may have determined that doing such big data analysis violated some international data protection laws."

Gabe Gumbs, vice president of strategy for Identity Finder LLC in New York, said the public nature of Twitter does limit its value to terrorists.

"Criminals use Twitter for a variety of nefarious activity. Terrorists are known to use Twitter for spreading propaganda and recruiting. Hackers have used the platform for command and control communications of botnets," Gumbs said. "Twitter is no more attractive than, say, Facebook for criminal activity; nonetheless, it is used. The government has a legitimate interest in access to posts made on Twitter and have been able to use information from Twitter during criminal and terrorist investigations."

However, Herold noted that U.S. intelligence agencies likely have other ways to gather data from Twitter without access to the firehose.

"The NSA certainly could create their own big data analytics program -- they probably have already done so -- to scrape the tweets and perform their own analysis," Herold said. "They have access to all the public tweets just like everyone else, and they certainly have the resources to do this."

Gumbs agreed, but said there is a big difference between the public APIs for Twitter and access to the Twitter firehose.

"The difference between the streaming API and firehose is that the streaming API only filters based on defined filters. An example would be an intelligence agency filtering post, in real-time, that had the phrase 'dirty bomb' in the post. All other posts would not be captured," Gumbs said. "The firehose, in contrast, gives access to every single post made in real time, allowing that information to be saved and searched at a later time, fed to another system for correlation and generally allows for anything to be done with the data."

Next Steps

Learn more about the effects of the EU General Data Protection Regulation.

Find out if government surveillance is going too far.

Get tips on how to protect corporate data from government surveillance.

Dig Deeper on Information security policies, procedures and guidelines