Gartner created its first market guide report on the network traffic analysis tools category in February 2019. These tools, the report explains, possess features that allow them to detect suspicious traffic that other security tools miss. Gartner's report includes a list of some vendors whose products have the following characteristics deemed central to this new type of security tool:
- They operate on network packet captures or flow data -- e.g, NetFlow, sFlow, IPFIX -- in near real time or better.
- They use network behavioral techniques like machine learning (as opposed to signature-based detection).
- They operate in the detection -- not the forensics -- phase.
- They analyze both north-south and east-west traffic and report suspicious events.
There were also some restrictive exclusion criteria that have been the basis of older generations of security tools, like working from log files and primarily using rules and signatures.
How network traffic analysis tools work
Network traffic analytics security can be achieved in a variety of ways. Machine learning, a simple form of AI, is one of the key components of network traffic analytics tools because of the way machine learning automates response to a perceived issue. It is critical that network traffic analysis tools work in real time -- or near real time. New forms of malware can spread very quickly within an organization, following both north-south (client to server) and east-west (server to server) network traffic paths. Automated traffic analysis and action is needed to prevent the rapid spread of new malware.
Some organizations are still skeptical of automated response systems, preferring to use manual methods for reacting to security events. However, the rapid distribution of ransomware is faster than manual methods can handle. This fact will eventually force all organizations to switch to fully automated reaction and remediation systems. An adaptable artificial intelligence or machine learning mechanism in any network traffic analytics tool is preferable because the variations in malware attack and distribution mechanisms causes a heuristic-based system to become outdated very quickly.
NIST's Cyber Defense Matrix
Tools that perform network traffic analytics cover an array of functions. One way to understand these tools' capabilities is to use the Cyber Defense Matrix product overlay and view network security systems. The relevant security function is mapped across the horizontal axis of the matrix, and the relevant infrastructure component is displayed down the vertical axis. The red outlines in the chart below indicate the general areas that are handled by the tools that perform network traffic analytics.
Selecting a network traffic analysis tool
There are several things to consider as you go about investigating network traffic analysis tools. How does each tool monitor traffic? Does it need full packet capture, or can it work from flow data, such as NetFlow, sFlow or IPFIX? How many collection points do you need? Some networks have a centralized topology in which there are a few key traffic aggregation points, while other network topologies will be more distributed and require many traffic collection points. Determine if network packet brokers or other technology will be needed to collect the necessary traffic information and direct it to the network traffic analysis system. A network packet broker works to aggregate data flows and then distributes them to relevant tools (e.g., security tools or monitoring tools). This aids visibility and coverage in the data center.
You should find out if you will need to provide feedback to the network traffic analytics system's machine learning algorithm. In some cases, you may need to provide acknowledgement that an anomaly was correctly identified or that a reported anomaly was a false positive. Make sure that you clearly understand the level of expertise and the amount of effort that you will need to supply in order for the deployment of the network traffic analysis tool to be successful. Of course, you'll also need to check price, including the effort required to implement an open source network traffic analysis tool.
The Gartner report includes a long list of commercial products. Open source tools may offer an alternative if you're willing to invest a lot of time to understand, install and maintain them. Note that many open source network traffic analysis systems are a collection of several tools that gather, index and store traffic data and perform analysis on it. One system that seems to meet the Gartner criteria is Suricata. The Google Rapid Response incident response framework might be another possibility, if you have a very large infrastructure, although it might not exactly match the criteria of a network traffic analytics system.
The increase in computing power and decrease in its cost has brought advanced technologies like machine learning into reach of many applications. Network traffic analysis is one such application. There are also a number of open source machine learning platforms, making it easier for companies to incorporate some form of machine learning without having to build their own software from scratch. Expect to see better results as the technology matures, including the development of more open source network traffic analysis tools.