alphaspirit - Fotolia
Threat intelligence services may be all the rage this year, but you could argue that this sort of thing has been around since 1997, when Internet pioneer Paul Vixie created a real-time blackhole list containing IP addresses that sent spam. Right out of the gate, Vixie (now CEO of Farsight Security Inc.) solved a problem that threat intelligence providers still struggle with today -- automated updates (though the "real time" of the '90s probably isn't quite up to today's pace). The RBL published by MAPS (Mail Abuse Prevention System) used updates to the Border Gateway Protocol to route spam sources to "nowhere."
The front-running antivirus and antimalware products at the time also began to offer automated updates to their malware signature libraries. The latest signatures were created from viruses found by "sensors" in the wild (that is, those first unlucky victims of a virus who sent a sample to their vendor to sort out).
All of this might lead you to wonder why "threat intelligence" isn't something that's sent along to current security products on an automated basis as part of the deal struck when the product was purchased: Why shouldn't your next-gen firewall soak up regular threat intelligence updates? Companies like Juniper Networks Inc. and Palo Alto Networks Inc. are moving in that direction, with subscription-based security services (including global threat intelligence), but to date, that's not how things have generally worked. Deciphering the volume of threat information and figuring out if and how it applies to your organization in a timely fashion is a big part of the problem.
Instead, threat intelligence has sprung up as separate data sources or exchange services, offered by member-based communities (crowdsourcing), vendor's aggregated threat intelligence clouds (Palo Alto Networks and FireEye Inc.) and a wide range of global, regional and niche services from industry leaders and startups alike. Some of the information is very sophisticated (cue the big data analytics) and some of it is open source mailing lists. An estimated $250 million threat intelligence market worldwide in 2013, according to Gartner Inc., is expected to reach $1.5 billion by 2018.
Intel wish list
Preliminary SearchSecurity research points to a major problem with the current offerings: Unlike antivirus signature updates, many threat intelligence feeds require a fair bit of digesting before anything useful can be done with the data. It's hard to make sense of continuous threat updates in time to make meaningful use of them. Our May 2015 survey of 276 European security professionals indicated that their No. 1 priority for threat intelligence services is receiving real-time updates (59% rated this as one of their top three features).
One in three respondents ranked integration of third-party intelligence feeds as one of their top priorities. But as The MITRE Corp., a federally funded non-profit research and development organization, put it in a description of their standardization initiatives, using this stuff is "either a time-consuming, manual process or a limited-scope automation effort tied to particular cyberthreat information-sharing community or technology." MITRE has spearheaded two interrelated efforts: The Trusted Automated eXchange of Indicator Information (TAXII) lets threat intelligence services speak a common language, while Structured Threat Information eXpression (STIX) provides a common vocabulary for describing the technical aspects of threats. In theory, you use TAXII to exchange threats that are notated in STIX.
Even with these standards, there's the question of where to apply the information from the feeds. Should next-gen firewalls understand TAXII and adjust their rules accordingly? A problem here is that some feeds provide enormous amounts of cumulative data -- IP addresses that have been used as bots, for example, numbering in the millions.
One innovative approach is the use of purpose-built appliances that are designed to offload some of the traffic monitoring and cyberthreat assessments and help other perimeter tools, such as firewalls or SIEM systems, respond faster by blocking traffic or sending alerts (fewer false positives) based on a vendor's intelligence feeds. A recent example is the box that Norse Corp. builds for its Intelligence Network. The dedicated 1 Gbps appliance (formerly called DarkWatch) was introduced last August and the 10 Gbps version, available as a rackmounted or virtual system, followed in April.
Jeff Harrell, vice president of marketing at Norse, says the appliance clarifies the value of an intelligence feed. "It's difficult to explain to people the difference between the free feeds you can get out there and something like Norse ThreatList, but it makes it a lot easier when you can put something like the Norse Appliance in and just watch it block things."
The Norse feed, among other things, provides a list of some 8 million IP addresses that have shown themselves to be potentially malicious, with the data coming from a network of sensors spread worldwide across 50 companies and 200 data centers. The network, which Harrell says is basically a Tier 1 backbone that exclusively serves its several million sensors, contains emulations of just about every sort of device you could imagine, from complex servers to wireless baby monitors. There's a 7 petabyte database of historical data on all the IPs they track, with 150 terabytes of new data arriving daily.
In other words, developing good data intelligence can be a difficult proposition. That hasn't stopped an impressively large number of vendors from entering the fray. We count upwards of 30 at present, ranging from BAE Systems to Webroot Inc. Pricing models for these services are still a work in progress, but you can expect to part with significant coin for more elaborate offerings. Norse ThreatList, for example, starts at $150k, if you go with the appliance approach. It's a lot, but at least with an appliance you know the data is being put to work on a continuous basis and, with any luck, you'll be able to incorporate feeds from multiple sources on the same device.
About the author
Robert Richardson is the editorial director of TechTarget's Security Media Group. Follow him on Twitter: @cryptorobert.