lolloj - Fotolia

Manage Learn to apply best practices and optimize your operations.

Network security: Threat intelligence feeds parse a sea of data

Threat intelligence feeds help you prioritize signals from internal systems against unknown threats. Security intelligence takes it a step further.

Many enterprises have made substantial investments in security information and event management and log management technologies over the years to collect, manage and analyze logs. Advances in large-scale analytics enable a well-honed security program to use data to spot anomalies and analyze attacks. However, it is feasy to be overcome by a deluge of indicators and warnings derived from this data.

Security intelligence promises to bring more focus to this task. Instead of blindly looking for "new" and "abnormal" events, we are now able to search for specific IP addresses, URLs or payload patterns. This is particularly important because compromises remain undetected for long periods of time; most companies are notified by external entities, not by internal sensors -- despite advances in data collection and event monitoring.

Today, the number of threat intelligence feeds continues to expand, from free open source data provided by the larger network security community; to vetted and aggregated commercial products; to closed information-sharing communities -- which are specific to particular industries or focus areas. When you are implementing and using these threat-intelligence feeds, you have to be careful not to, once again, drown in data. Security intelligence is supposed to make it easier to pick out signals in the ocean of log data, not raise the noise and analytic overhead during ongoing log analysis.

Security intelligence is supposed to make it easier to pick out signals in the ocean of log data, not raise the noise.
Johannes B. Ullrichchief technology officer, SANS Technology Institute

Organizations need to consider three factors before selecting the best security intelligence feeds for their business risks and priorities:

  • Business alignment. What types of threats does the business face? And which ones are insufficiently captured by internally generated security intelligence products?
  • Sensor capabilities. What data are you able to collect, and what sensors do you have in place or plan to deploy?
  • Intelligence gap analysis. Which current threats to the business could be mitigated more efficiently if you had a particular intelligence feed available?

A financial services institution may have to mitigate threats, not only against its own infrastructure but also threats such as banking malware that affect its clients. A threat intelligence feed of recent command-and-control servers used by this malware may be less useful compared to intelligence about artifacts in HTTP requests. The business is unlikely to be able to monitor customer equipment connecting to the command-and-control server, but a feed of the IP addresses of compromised machines will be useful to identify transactions placed by these systems and to notify infected customers.

Threat intelligence should not be limited to simple network and host-based signatures like IP addresses and hashes of malicious software. To align security intelligence with business problems, consider transaction volumes or industry-specific performance indicators relating to transactions and other risks. For global organizations that are primarily concerned with foreign competitors attempting to access intellectual property or trade secrets, the capabilities of these potential adversaries, their geographic locations and past methodologies need to be considered.

To reach its true potential and help organizations fully understand internally collected data, threat intelligence also needs to be shared. Information from other organizations, particularly those in similar industries, is helpful as they likely face similar threats. But sharing your threat information is helpful as well -- it may provide additional context that was not evident from the data you collected internally. An attack that was only considered partially successful against your network may have been effective against other organizations' networks. Those network security teams will now be able to share more details as to the intent and capabilities of the attackers, allowing your team to go back to your systems and search for additional indicators that they may have missed.

In the end, a security intelligence program is only successful if it produces relevant, deployable signatures that aid the business by mitigating threats more effectively.

About the author
Johannes B. Ullrich, Ph.D., GIAC, GCIA and GWEB, is chief technology officer at the SANS Technology Institute and head researcher at its Internet Storm Center. Follow him @johullrich.

This was last published in August 2014

Dig Deeper on Malware, virus, Trojan and spyware protection and removal