Problem solve Get help with specific problems with your technologies, process and projects.

Ain't misbehavin': Security tools watch behavior to stop new threats

Robert Scheier takes a look at behavior-based and signature-based security tools.

Who would you rather have screening the fellow passengers on your plane: A security guard who is only checking for the names of known terrorists, or a security guard who is looking for suspicious behavior from anyone?

Ideally, of course, a security guard should watch for both things. And that's the idea behind "behavior-based" security tools, which monitor the actions taken by files (on a server or PC) and packets (on a network) and raise a red flag if those actions look suspicious. The best defense, according to analysts, is a combination of behavior-based security tools and the "signature-based" approach used by most antivirus software. Indeed, a number of host and network-based intrusion-detection systems rely on both signature-based and behavior-based protection.

Sign here

Signature-based tools compare files or packets to a list of "signatures" of specific files or packets known to represent a threat. (Each signature is the specific arrangement of zeros and ones that make up a file.) Behavior-based tools compare the behavior of files or network packets to a list of accepted (or of suspicious) activities and take action (either blocking the activity or generating a warning) if they see a behavior that looks suspicious or is forbidden.

In general, signature-based tools are best at identifying and repelling known threats, while behavior-based are best for fighting new threats that haven't made it onto a list of known threat signatures. Most behavior-based tools come with a standard set of policies for which behaviors are allowed (or are suspicious), while also allowing administrators to create their own policies.

Some behavior-based tools operate on servers or PCs and usually examine calls, or requests, from applications to the operating system and compare them with a list of accepted (or forbidden) behaviors. These include StormWatch 3.0 from Okena Inc. and Harris Corp.'s Stat Neutralizer. Some tools specialize in protecting Web servers, including eEye Digital Security's Secure IIS, Entercept Security Technologies' Web Server Edition (which combines behavior-based and signature-based protection), Pelican Security Ltd.'s WaveBreaker and Sanctum Inc.'s Web AppShield 4.0.

Other behavior-based tools work on networks, examining traffic flow and looking for anomalies such as unusual traffic to or from a certain IP address, a port on a server or an application. They include Lancope Inc.'s StealthWatch appliances and IntruVert Network's IntruShield, which combine signature and behavior-based monitoring.

Some tools span both servers and networks, such as Internet Security Systems Inc.'s RealSecure Protection System. Finjan Software Inc. uses behavior-based monitoring in its SurfinGate tools for e-mail and Web gateways and its SurfinShield software for corporate PCs, but also bundles the McAfee Security signature-based antivirus product into its products.

Ted Doty, director of product management for Okena, claims that behavior-based tools, which run on a PC or server, can find many of the same threats as signature-based antivirus tools. That's because many viruses attempt the same sort of malicious behavior, he says, such as to "open the Outlook address book to send outbound mail to everyone in the address book. If you're intercepting operating system calls, it's very easy to see and prevent" attacks such as this.

Unlike antivirus tools, which look mainly at contents of files, some behavior-based tools can also examine malicious Java scripts or executable files that can be embedded in the HTML stream downloaded by a Web browser.

In a different approach to "behavior-based" security, Authentor Systems Inc. examines users' behavior (such as when, how often or from where they log in) to ensure they are who they say they are.

Neither signature nor behavior-based tools are silver bullets, observers say. Antivirus tools are a useful complement to behavior-based tools, according to Doty, because they can perform follow-up work such as "disinfecting" a system by removing or quarantining the viruses. And while behavior-based security tools are better than signatures are at stopping new threats, says Pete Lindstrom, Research Director at Spire Security, they could keep users from doing legitimate work if they're set to block too many types of behavior. That's why he recommends choosing tools with robust monitoring and logging capabilities so administrators can analyze behavior on the network before they block it.

"In theory, one type of (security tool) without the other would be sufficient," says Lindstrom. But practically speaking, he says, "you need both."

About the author:
Robert L. Scheier writes about security from Boylston, Mass. He can be reached at

This was last published in November 2002

Dig Deeper on Network intrusion detection and prevention (IDS-IPS)

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.