'Targeted' perimeter defense improves network-based intrusion detection systems

Target-based IDSes squelch network noise to pinpoint the alerts you really care about. We review three solutions to see if they hit the bull's-eye.


Click to enlarge.
Doubleclick to restore.

Network-based intrusion detection systems (NIDSes) are at least as famous for their failures as their successes. Most NIDSes generate a lot of false positives, false alerts, false alarms--whatever you call them, they've driven many frustrated security managers to simply pull the plug on their NIDSes. Gartner Group went so far last year as to proclaim that "intrusion detection is dead."

The vendors' response is smarter NIDSes. "Target-based IDS" is a new technology that correlates knowledge about network topology, operating systems and applications with incoming attack information.

That appealed to Ed Goff, whose North Carolina-based power utility -- Progress Energy -- was in the market for an IDS solution for its internal networks, to complement its perimeter defense.

"Using the strategy of correlating bad activity sensed on the wire to our hosts enabled us to leverage the intelligence of the application instead of working so hard and relying on human elements to see trends," says Goff, who chose an Internet Security Systems solution. "We can now aggregate and correlate what we are actually vulnerable to."

We looked at cutting-edge products from Cisco Systems, Internet Security Systems and Tenable Network Security, which are designed to make the alerting side of NIDS consoles smarter and more reliable. The results look good: You'll have far fewer irrelevant alerts, cutting your time to analyze alarms from hours to minutes each day. But, like any filtering system, target-based IDS has its own "false positives," occasionally omitting important information.

These products combine traditional network scanning and vulnerability analysis with IDS alerting consoles. For example, by matching an alert for SQL Slammer against knowledge of whether the targeted system is running SQL without the relevant patches, a target-based IDS can apply the appropriate priority level to the alert.

Information Security invited several IDS vendors to participate in this bakeoff if their products incorporated "target-based" information, either at the signature level or as a filter on alerts.

Three vendors accepted our invitation -- Cisco Systems, Internet Security Systems and Tenable Network Security.

Symantec, Network Associates, NFR Security, Sourcefire, Computer Associates and Lucid Security declined our invitation, either without comment or because they don't have a solution that fits our profile.

We evaluated target-based IDS solutions based on five key criteria:

  1. Network/application information collection tools. Are vulnerability scanners used, or does the console gather information? What network topologies and sizes can the scanners handle? How much preloaded information does the scanner require?
  2. Information collection timing/trigger. Is it gathered prior to an alert (active or passive)? Or is it gathered after the alert occurs?
  3. Evaluation method. Does target information modify the behavior of IDS sensors (i.e., prior to alerting)? Is target information used to promote or demote alerts (i.e., after alerts have passed to console)?
  4. Vulnerability detection technique. Are vulnerability scanners active? Are they passive? How much control do you have over vulnerability detection?
  5. IDS sensor support. Does the solution support only its own sensor(s) or other vendors' products as well?


Ron Gula, CTO and founder of Tenable, describes IDS alerts as "raw intelligence." An analyst can convert this data into "well-qualified intelligence," but analyses take time, and they're expensive. Automating the process of qualifying alerts is a big piece of what target-based IDS is all about. Give a computer rules for promoting and demoting alerts, and you can turn 130,000 events (the average daily number we saw in our testing) into a half-dozen that really need attention.

The target-based IDS products we reviewed do only one thing well: determine whether a system is vulnerable to an attack identified in a particular signature. They combine a normal IDS engine with post-processing tools to convert alerts from "raw" to "well qualified." Alerts flow into the console from the NIDS engine in all their verbose glory. However, before a console displays alerts, they are promoted, demoted or otherwise qualified to reveal which systems are actually vulnerable.

We looked at three target-based IDS tools -- Cisco Threat Response, ISS's Fusion and Tenable's Lightning Console--to see how mature this technology is and how well it solves the problem of IDS noise. Each tool was installed with its accompanying IDS sensor on our production data network, then tuned by the vendor. We let each run for two months.

The architectures were basically similar. Each tool has three main components:

  1. A network scanner to collect and manage vulnerability information. The scanner maps hosts on the network, the services each is providing, and their versions and patch levels.
  2. A network of standard, off-the-shelf IDS sensors.
  3. A console that matches the alerts from the IDS sensors against the vulnerability information from the network scanners to help qualify alerts for the security analyst.

That said, there are some significant differences, which reflect the immaturity of this technology:

Cisco. The Cisco Threat Response (CTR) (see Figure 2) product line was part of Cisco's acquisition of Psionic Software in late 2002, and is in a state of flux. We looked at CTR as a standalone product, which adds on to Cisco's existing IDS sensor Management Console and VPN and Security Management Solution products. CTR supports Cisco's IDS sensor line, including the Cisco 4200 series, which we used, as well as the ISS RealSecure sensor.

CTR has a major architectural flaw, which Cisco is correcting by pushing CTR technology both into their sensors and existing management console. CTR's network scanner and vulnerability collection tool and management system all run in one box. This makes collection of vulnerability information and network scanning difficult in a large network, especially in environments where many LAN segments are linked using both routers and firewalls.

CTR is typically deployed in a network operations center, topologically distant from the IDS sensors. The view of the network, through firewalls, routers and NAT devices, may be very different on the CTR console than where the IDS sensors are located.

CTR's core behavior is reactive, which is unique among the tested products. In other words, CTR doesn't attempt to determine if a target is vulnerable until it receives an alert. When you set up CTR, you tell it how long to "remember" information it discovers. You don't want to remember things forever because hosts change over time. With CTR, there's a delicate balance between not overwhelming your network with active data gathering and not holding onto old data. You can also periodically launch network and vulnerability scanning to gather information.

However, not everything can be discovered ahead of time. For example, when CTR sees some types of Web-based attacks, it downloads and analyzes Web logs from the targeted system to determine whether the attack was successful.

ISS. The Fusion alert evaluation tool (see Figure 3) promotes or demotes alerts from its IDS sensor--in this case, ISS's Proventia Appliance -- based on information gathered by ISS's Internet Scanner. The solution consolidates management through ISS's SiteProtector console, which controls NIDS sensors, host-based IDS sensors, desktop security tools and Internet Scanner.

The ISS toolkit was unwieldy to set up, but it has tremendous scalability and flexibility in placing sensors and scanners throughout a large enterprise network.

Fusion brings two main features to the table. First, it uses information retrieved from Internet Scanner to assess the likelihood of a successful attack. We found the accuracy here to be very high.

"On a scale of 1 to 10, I rate it 8 for accuracy," says Progress Energy's Goff. "It's where the rubber meets the pavement. If I'm looking at a high alert, and Fusion tells me it's the wrong OS, I can ignore it."

Second, Fusion recognizes attack patterns by looking across multiple events. It includes approximately 20 attack patterns, such as "attack from compromised host," in which a compromised host can become a launching pad for another attack.

But this security professional's dream -- automated event analysis for patterns -- is plagued by errors. The pattern analyses were wrong more often than they were right during our testing. For example, Fusion kept erroneously reporting -- several times a day -- that our SMTP and DNS server was being attacked with a buffer-overflow attack on the SMTP side, although the specific signature would have resulted in a denial of service (crashing the SMTP process) rather than privileged access.

Tenable. Lightning Console (see Figure 4) works with third-party IDS sensors, including NIDS sensors from ISS, Network Associates and Enterasys Networks, as well as open-source Snort and Bro. Lightning Console can work with the open-source Nessus scanner, or its NeWT (Nessus on Windows) or NeVO (its passive scanner), both of which we used. And, we ran Lightning Console against a Snort sensor.

The NeVO passive scanner strongly differentiates Lightning Console from CTR and Fusion. NeVO monitors traffic, discovering operating system and stack information, lists of open ports and application information (See "Passive Scanning, Let It Happen").

While CTR and Fusion watch alerts and then check for relevant vulnerabilities, Lightning Console starts with vulnerabilities, and watches attacks to see if any match up.

Lightning Console classifies each event as either vulnerable or not. It's a very black-and-white definition, based exclusively on the target network's known vulnerabilities. This approach has its drawbacks. For example, when our Snort sensor started to generate alerts about decoding problems, Lightning Console demoted them because they weren't mapped to a detected vulnerability. If you get into the habit of not looking at "not vulnerable" alerts, you'd miss a critical problem. More seriously, if a signature doesn't have a matching vulnerability, it won't show up in the "vulnerable" list. This means that if your scanner, such as Nessus, hasn't kept up with your IDS, you may miss alerts. Tenable also has an automated tool for mapping IDS signatures to vulnerability information that can be confused if the IDS signature isn't well formed.

Lightning Console can't be used to manage your third-party IDS sensors, but it does manage the vulnerability scanners Nessus, NeWT or NeVO.

So, does this first wave of target-based IDS technology actually reduce the noise coming from the NIDS console? The short answer is "yes," though there are some limitations.

All three of the IDS consoles had a substantially lower noise level once properly configured as target-based IDS.

ISS. Fusion, through SiteProtector, is more of a prioritization tool than a list of must-review alerts. Unlike CTR and Lightning Console, ISS didn't change its user interface or capabilities when it incorporated Fusion's assessment and correlation capabilities. Fusion has a four-level rating scale: "Success Likely," "Failure Likely," "Attack Failed" and the all-important "Don't Know." SiteProtector, with a wide variety of powerful filters, and drill-down and pivoting tools, offers almost unlimited options for slicing and dicing events. Compared to CTR and Lightning Console, Fusion has a definite edge as an analytical tool.

Tenable. The quietest console was Lightning, which routinely boiled down tens of thousands of events to almost nothing. In our testing, Lightning flagged just one event type: remote downloading of the robots.txt file from our Web servers (robots.txt is the file search engines retrieve before scanning a Web site to see which directories they should ignore). There's a big caveat though, because of Lightning's strict criteria for promoting events. Lightning won't show you an event unless there's a known vulnerability on the target system. Moreover, the definition of vulnerability is highly dependent on how you tune your VA scanner. For example, anyone connecting to Windows Terminal Services will set off an alert in Snort, and Lightning Console, in turn, will issue a warning. The issue here isn't that Terminal Services has a specific known vulnerability, but that the authors of Nessus believe that the whole idea of Terminal Services is potentially insecure enough to warrant calling out more forcefully than other remote access protocols. Fortunately, for sites making heavy use of Terminal Services, you can tune the VA scanner to not be so biased.

Thus, Lightning is a great tool for managing vulnerability-based alerts, but does nothing for other aspects of IDS usage, such as policy-based alerts.

Cisco. CTR shows a rolling window of the last 1,000 events, color-coded red and green. On our network, 1,000 events popped up about every 30 minutes. At any one moment, we'd have about 10 different "red" type events, representing a couple of hundred potentially dangerous events.

CTR takes a more conservative approach to noise reduction, resulting in a larger list of critical events than Lightning Console. While Lightning Console only shows alerts linked to vulnerabilities on the target system, CTR displays everything. For example, when someone swept through our Web servers trying a known attack on Lotus Domino, CTR sorted them into 22 green alerts and one red alert for a probable success on a Domino server. Having the ability to drill down to see attackers gave us a better idea what was happening on the network. Was this a random single attack? A sweep by a script? Many attackers coming from many directions? CTR trades off a very tight list of known problems by giving security analysts more contextual information for forensics investigation.

We saw huge differences in philosophy on the critical point of how vulnerability information is gathered. Every aspect of vulnerability scanning is up for debate, from where to put the sensors to how often to scan to whether to scan at all.

ISS. Fusion depends entirely on periodic active scanning with Internet Scanner, compared to CTR, which launches targeted scans when it receives an alert, and Lightning Console, which uses passive as well as active scanning.

Internet Scanner is a sophisticated tool, but it's a very traditional active scanner with the associated limitations--scanners are invasive tools that can slow and disrupt network operations, and so they must be used selectively, at the least disruptive times. More important for targeted IDS, your vulnerability information is only as current as your latest scan. The biggest example we saw was the recent W32/Blast worm. A system infected with W32/Blast will start up a TFTP server to help propagate the worm. Because the TFTP server wasn't there when Internet Scanner went looking, Fusion marked the TFTP alert as "Attack Failed," even though the sudden appearance of a TFTP server on a system that never had one is the best sign that a W32/Blast attack had actually succeeded.

Cisco. CTR is more about vulnerability verification than proactive scanning. While elective scanning is an option, CTR is designed to launch a scan when an alert comes in. CTR immediately launches its proprietary scanning toolkit to determine whether the alert applies to the target host. It will attempt to determine operating system and version information, as well as application version number. For many applications on Windows servers, CTR will use a username/password the security manager has provided to log into the targeted system to verify patch levels as well. With some attacks, CTR goes even further: It will download log files to verify that an attack was successful against a Web server.

CTR caught the TFTP server that ISS missed, and for some classes of events, especially Web-based ones on Windows servers, CTR's ability to connect to the server under attack and pull down log files is impressive. This level of automated research to answer the question, "Did this attack succeed?" is far beyond the capabilities of Fusion and Lightning.

One caution: You must give CTR username and password information for your servers to accomplish this magic, which many system managers may be unwilling to do.

However, waiting until an attack comes to check for vulnerabilities can generate its own pathological behavior. If your CTR box is firewalled from a system it's scanning, it can be a denial-of-service source by persistently attempting to find out information it can't access. This problem is exacerbated by the current CTR technology, which fixes the scanning engine at the console.

In addition, an attack that covers up its own traces or patches the hole it exploited might be demoted if CTR can't find evidence of the vulnerability after the fact.

Tenable. Lightning Console has a much more aggressive approach to vulnerability information collection, offering both the Nessus (and NeWT) active scanners, as well as its NeVO passive scanning. This gives the network manager a choice of strategies: traditional active scanning, passive scanning or a combination of the two.

Passive scanning is less comprehensive than active scanning, but it's also less intrusive, and the capability to combine both techniques in the same product is a valuable part of Lightning's architecture.

Tuning -- the most difficult, time-consuming, and frustrating part of IDS management -- is completely ignored by the target-based products we looked at. They apply their intelligence on the output side of the IDS sensor -- breaking down the noise. This intelligence would be useful in IDS sensor configuration as well, but, none of the products have a good feedback loop in their sensor configuration.

Some of the vendors waved this issue away, saying it's best to filter alerts on the output side. Our experience with performance says otherwise. All of the consoles were heavily burdened by the number of events that our little test network (fewer than 250 systems) was generating, reducing the total number of alerts generated would be useful.

Actively providing feedback from scanning to IDS sensors is critical in reducing noise and in making NIDS smarter. IDS sensors don't apply all their signature logic to all traffic on all ports of all servers. Instead, they tend to look at well-known ports. For example, SMTP-based signatures are typically run only against port 25. An active scanner that discovers SMTP running on nonstandard ports should be able to feed that information back to the IDS, but these products don't. It's a glaring shortcoming.

Target-based IDS also has self-tuning requirements. Because all vulnerability scanners can and will make errors, it's critical that the network manager be able to both add and delete from the vulnerability information discovered by the scanners.

Cisco. CTR has limited tuning capabilities. For example, we knew that our OpenVMS systems weren't vulnerable to a particular attack. With CTR, you simply delete the operating system from the list of those susceptible to the attack. However, you can't say, "this particular system isn't vulnerable to this alert." This is because CTR is designed around OSes rather than applications, and the user-defined policies are at the OS level. If you have a global operating system, such as Windows 2000, but use different Web servers throughout your data center, CTR would be particularly frustrating.

ISS. Fusion doesn't offer tuning of the target-based aspects of the system. Because the policies used to map vulnerabilities to alerts are opaque, you don't get any opportunity to modify or control them. While you can change the number, level and sophistication of scans very easily, it's hard to use that control to modify Fusion's behavior.

Tenable. Lightning Console's tuning tools are the easiest to use, but not very granular. You can navigate to a particular vulnerability and mark it as a false positive for a particular system, for all systems in a particular location or of a particular type. However, other simple operations, such as "ignore SNMP alerts if they are generated by our management station," aren't possible.

The issue of tuning brings up another difficult question: "How does the target-based console know whether something is vulnerable?" Clearly there's a mapping between vulnerability scanning and alerts. How is this done in each product?

The answer is you've got to trust the vendors, because, for the most part, this is their secret sauce. For example, Lightning Console maps vulnerabilities to alerts entirely automatically, depending on well-formed alerts coming from the IDS sensor. Similarly, we found the policy language and descriptions in Cisco's CTR to be sloppy in many cases. ISS draws a dark shroud around Fusion's internal operation. Some pieces, such as how Fusion correlates events, are visible in the policy module, but the more mundane mapping of alert to vulnerability is completely hidden and can't be modified or tuned.

Target-based IDS is in its infancy. Cisco's CTR is in transition, with the technology being taken out of a standalone system and pushed into their sensor and management consoles. ISS's Fusion is a huge help in reducing alerts, but is still missing a vital tuning element. And Lightning Console, while a great technology demonstration of the power of combining active and passive scanning, has an immature user interface that needs major revision to have the ability to manage IDS sensors in its own console.

We also found that products ready for market are focusing on the post-processing of alerts coming out of the IDS sensor. Other aspects of target-based IDS, such as feeding host and network information to the sensor itself, are much less mature.

Finally, the scanning aspect of target-based IDS is a serious issue. Some enterprises simply can't allow active scanning technology on their networks, for political or technical reasons, or both. While Lightning Console offers the option of passive scanning, it's new technology that may not be as effective as active scanning.

Still, if you want to use the alerting function of your IDS system, target-based IDS has an undeniable benefit. In all cases, we saw a significant decrease in the amount of noise, helping us focus more quickly on the alerts that matter.

Joel Snyder is a senior partner at Opus One, an IT consulting firm.

This was last published in January 2004

Dig Deeper on Network intrusion detection and prevention (IDS-IPS)

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.