Published: 01 Nov 2007
Yet desktop security technology is rapidly advancing, with host-based intrusion prevention systems (HIPS), personal firewalls and other defenses augmenting traditional antivirus and antispyware tools. Because of the severe constraints on the number of host security products our enterprises can deploy and manage, major security vendors have responded with integrated endpoint security suites, rolling a bunch of desktop defenses into a single package.
These endpoint security products have introduced a new dynamism into our industry, as antivirus vendors augment their wares with fresh features to compete against each other and hungry challengers. To help sort out all of this, Information Security evaluated seven enterprise endpoint security solutions. We graded each on its management capabilities, reporting, ability to detect and block malware, detecting and thwarting exploit attempts, and integration of the various desktop security capabilities in one package.
Specifically, we tested CA Threat Man-ager 8.1 and Host-Based Intrusion Pre-vention System 8; eEye Digital Security Blink Enterprise Edition; IBM ISS Pro-ventia Desktop Endpoint Security 9.0; McAfee Total Protection for Enterprise; Sophos Endpoint Security and Control 7.0; Symantec Endpoint Protection 11.0; and Trend Micro OfficeScan 8.0.
Bearing witness to the rapidly evolving nature of the endpoint security space, the three giants of the information security industry--McAfee, Symantec and Trend Micro--responded with beta versions of their suites that were nearly finalized for shipping. (We requested every product we analyzed be available for general purchase by our publication date.)
Many of the problems we encountered with testing and, in some cases, retesting updated versions of these products reflected the difficulties in dealing with beta builds of highly complex packages. But, further, our testing suggests this class of integrated endpoint security products is, for the most part, far from mature.
The immense complexity of these tools can be overwhelming, with more features than almost any distributed system in today's enterprise. If a given product provides really good security, but cannot be managed across an enterprise in a coherent fashion, it just isn't useful.
We looked at the ease with which each product could be used to configure systems, quickly determine their security state, and update settings based on attack activity.
We were particularly impressed with Symantec's management capabilities, but McAfee's completely new ePolicy Orchestrator (ePO) is a major disappointment.
|Symantec is top-notch from a large enterprise perspective, with intuitive GUIs for policy configuration and status checking. Its overall dashboard clearly identifies potential problems associated with infection, out-of-date signatures or disabled functionality on managed hosts, offering advice to an administrator on how to fix each issue. The management GUI comes in two flavors: a full-blown Java-based interface for all aspects of the administration console, and a scaled-down Web-based interface that can be used for status checks and reporting, but not policy management.
Sophos also provided very solid management capabilities, organized, the vendor told us, around the KISS principle, which we assume stands for "Keep It Simple, Sysadmin." Sophos' GUI is designed to reduce the time and effort needed to configure and deploy the product. Sure, you don't have access to a lot of the fine-grained policy settings, but the overall options available for configuration are excellent. Checking the status of managed workstations is snappy, and alerts about systems that deviate from policy are easy to understand.
The Trend Micro management interface worked quite well in configuring and analyzing managed systems, especially for antivirus and antispyware. The new product architecture enables Trend Micro to package new endpoint product building blocks into plug-ins for rapid deployment, a design decision that will benefit Trend and its customers.
However, we were concerned that we couldn't discern client signature updates for its Intrusion Defense Firewall (the component that implements the firewall and HIPS functionality). Such information is vital in signature-based IPS products such as this one, which applies network-based IPS signatures to traffic going into the protected host. Trend Micro licensed this functionality from Third Brigade to create the first plug-in for its new architecture.
eEye Digital Security features a well-organized, intuitive management interface. However, the client GUI is clearly more mature than the enterprise management console itself, offering finer-grained insight into the configuration and alerts generated by the tool.
|CA's management console has improved significantly since we last looked at it in our antispyware analysis in May 2006. Its latest version is much faster and more interactive than previous versions. Still, checking the status of different workstations required moving between different screens, and policy configuration of this purely Web-based GUI was more difficult than with other products.
We found the IBM ISS product quite difficult to manage. Determining the current status of clients from the management console was cumbersome, and managing all of the separate features was complicated and confusing. Also, at several points we encountered cryptic error messages that didn't explain the problems we encountered in installing and configuring the product. Finally, the IBM ISS endpoint product is exclusively for Windows clients; it cannot be used to manage servers, Windows or otherwise. Server security is available only as a completely separate product.
McAfee's new enterprise management server, ePolicy Orchestrator (ePO) 4.0, was a great disappointment. The more you loved the previous versions of McAfee's ePO, the more frustrated you will likely be with the new version.
McAfee has completely rewritten its flagship management product with a Web-based GUI, letting admins manage it from any browser in their enterprise.
The well-laid-out and quick GUI of earlier ePO versions has been replaced with a complex and bewildering Web-based interface. The easy drag- and-drop features of the thick-client ePO have been replaced with countless Web-based drop-down menus in screens that make it difficult to find what you need.
The difficult-to-use management GUI and default policy are of significant concern. While testing, we accidentally applied a baseline medium security policy to the McAfee management server itself. In the complex ePO 4.0 GUI, such mistakes are frustratingly easy to make--we did it while being guided step-by-step on the phone by McAfee support. By simply applying the default security policy to the management server, ePO killed itself. We were unable to get access to any of the management capabilities, and had to reinstall everything from scratch to resume testing.
To gauge each vendor's ability to detect and block malware found in the wild, we ran three tests using 8,114 recent malware specimens from a private collection graciously provided by antispam researcher Bill Stearns. Our zoo included a large variety of worms, bots, backdoors and viruses. For each test, we recorded the percentage of specimens not eradicated in each round of testing (See "Antimalware Scanning Results," PDF).
We then performed an on-demand scan of all malware that survived our first test, to assess the combined real-time and on-demand scan capabilities for identifying and eradicating malware.
Finally, we conducted on-demand scanning independently by disabling real-time scanning, copying all malware to the target file system, and then executing a scan of the entire zoo.
Trend Micro, CA and eEye all did very well, generally detecting and blocking or removing all but about 8 to 9 percent of the malware we threw at them in all tests.
|Symantec was close behind, missing 17.6 percent of specimens on the real-time scan, but performing on a par with Trend, CA and eEye in the on-demand scans.
McAfee was next, with 22.3 percent of our specimens eluding the real-time scan. The follow-up on-demand scan, however, produced some surprising results: Another 10.7 percent of specimens were detected, but none of those were deleted or quarantined. Likewise, in the pure-play on-demand scan, all of the 8,000-plus malware specimens survived, despite an avalanche of alerts. That's because this new McAfee product has a default action of alert-only for on-demand scans, in contrast to the competition and a departure from most earlier McAfee products.
With the help of McAfee support, we used the McAfee client to conduct an on-demand scan with a delete action, a process that requires several rather nonintuitive steps. After that scan, 11.6 percent of our initial specimens remained for both the on-demand scan following real-time scanning and the pure-play on-demand scan. Notably, McAfee blocks all .exe files from a network copy, even benign test files, due to another default setting. Such a feature is likely to cause problems in environments attempting to distribute programs via network file shares, and is certain to be disabled in some enterprises.
When we tested Sophos, every one of our specimens survived the initial copy because, by default, Sophos' real-time defenses only look at "read" actions, not "write" actions. Such an approach, possibly done to improve file system performance, prevents the malware from executing, but does not stop infiltration of malware into a file system. Sophos does offer an option for changing this default behavior.
In the end, both the real-time/on-demand combo test and the pure on-demand test left 36.7 percent of the specimens on the target machine.
Sophos' default behavior is to perform "in-place" quarantine, preventing future access of the file but leaving it in its current location. All the other products move malware to a separate quarantine directory or delete it. Sophos says its approach makes restoration of files misidentified as malware easier. If your antivirus tool makes false-positive matches on legitimate files, restoring access in their normal locations is a lot easier than scraping them out of a quarantine directory and finding their homes again. The Sophos tool can be configured to perform traditional quarantine or deletion.
|IBM ISS was rated lowest in this series of tests, crashing several times and scoring so poorly as to cause us to double-check that protection was enabled. IBM ISS leaves signature- based antivirus turned off by default, another indication that this product is typically used to augment another vendor's antivirus solution. IBM ISS has licensed BitDefender's antivirus and antispyware functionality in its endpoint suite, which we activated before starting our test regimen. The initial real-time test completed without the tool blocking a single file. According to IBM ISS support personnel, file copies across Windows network shares are not scanned, even with the on-write scanning option enabled. This stance mystifies us, considering that users could copy infected files on a file server back to their clients without any real-time protection.
The on-demand scanning was hardly better. The follow-up on-demand scan started off as expected, but halfway through the scan (according to the progress bar) scanning stopped and we were greeted with the message "Successfully Completed." However, the same GUI listed "Number of Files Remaining: 4,430" and we still counted 58.6 percent of our malware in the target machine's file system. This stop and start repeated several times during the scan. We re-ran this test several times, but 34.7 percent was the best IBM ISS managed in repeated on-demand scans.
Every vendor in our analysis claims to protect systems against exploitation using some form of HIPS technology. Different vendors use this term for a variety of disparate technical defenses (see "HIPS Hydra," below). Regardless of approach, we wanted to see how each vendor would fare against exploitation attempts in a series of three tests. We disabled each product's firewall component to focus the test exclusively on HIPS functionality.
|Our second test measured how well each product defended listening services on the protected system, particularly services associated with Windows networking. We attempted to exploit both the MSRPC DCOM buffer overflow flaw (MS03-026) and the LSASS buffer overflow issue (MS04-011). To add some variety to this network service testing, we attempted each exploit with both a standard command shell payload and the Metasploit Meterpreter shell, a more sophisticated and often harder-to-detect attack that provides specialized remote shell access running from within an exploited process. Our client-side and server-side testing relied on Metasploit Framework version 3.0, using all default settings except for the HTTP port for browser exploits, which we changed from 80 and 8080 to another number to simulate an attacker who tricks a client into clicking on a link with a port number in it.
Our third test was designed to look at how each vendor could defend against zero-day exploits of third-party applications. We created our own network-listening program with a buffer overflow flaw, and wrote some code to exploit it to give remote command-shell access on the target machine.
Overall, eEye performed best in detecting exploits. It was the clear leader in identifying client-side attacks, alerting on all of our tests, but by default did not block; it simply displayed the alert "Application Protection: Suspicious System Call." This default behavior could be altered to block such exploits, as a global switch applied to all such suspicious system-call detection. While blocking is the goal, concern over false-positive blocks makes eEye's default setting reasonable.
eEye successfully stopped all service exploits. However, in the process of blocking the MSRPC DCOM exploit, it killed the svchost.exe process, which made our Windows machines reboot themselves within 60 seconds. It is generally considered better to kill an exploited process rather than run the attacker's code, but re-booting could result in loss of valuable data.
eEye detected and alerted on our zero-day exploit; when we tweaked the configuration to an action of "Terminate Process," it blocked as well.
|IBM ISS came next. On the client side, it detected and blocked the VML exploit. However, the alert messages for the IE CreateObject and Firefox attacks didn't indicate that the product had detected the exploit action, only that it identified a Microsoft Windows shell banner passing across the network. An attacker could launch such an exploit without creating a banner, thereby dodging this form of detection.
IBM ISS identified and blocked all services-based attacks, with an alert that cited the specific exploit we used, the ideal behavior for the product under these tests.
It allowed our zero-day attack, again merely alerting to the presence of a Win-dows shell banner.
Sophos delivered reasonable performance in our client-side testing, alerting on two exploits as "Buffer Overflow" behavior, but missing the CreateObject exploit. The default action is to alert, but Sophos can be configured to block the attacks.
All of our services attacks were detected, but by default they were allowed through, giving the attacker control of the system. Sophos neither detected nor blocked our zero-day exploit.
McAfee detected and blocked our VML and Firefox exploits, but failed to detect our CreateObject exploit. McAfee detected and blocked all of our service exploits. For zero-day defenses, McAfee requires administrators to configure specific applications to be protected on a machine. By default, nothing other than specific Win-dows components is protected, so our zero-day attack went undetected. As an experiment, we configured McAfee to add zero-day protection to our custom vulnerable application. Unfortunately, our exploit still went undetected.
|Trend Micro and Symantec came next in our exploit testing. Neither identified nor blocked a single client exploit. Trend Micro support personnel indicated that the HIPS protection it licensed from Third Brigade (as well as the protections offered by other vendors) is often configured by default to look for browser exploits only on TCP ports 80 and 8080. Again, independent of our scoring, we tweaked our test to verify this claim, and Trend Micro did detect our attacks on those ports. Administrators can add lists of additional ports for browser and other HTTP-related defenses. Ideally, an admin would configure the endpoint security suite so it monitored for HTTP and HTTPS attacks on all ports allowed out through the enterprise's network firewall. In many organizations, unfortunately, the number of ports allowed outbound are rather high and change on a regular basis, making this synchronization of network firewall and endpoint security tool difficult.
Both Trend Micro and Symantec detected and blocked all of our services exploits, but neither detected our zero-day attack.
CA fared worst of the seven products in this series of tests, failing on most. It didn't detect or block any of the client exploits with its default security policy. Although not part of the scoring, we experimented with its "Restrictive Policy," which did block all of the exploits, but also prevented Firefox from accessing the network.
The next set of results were, if anything, poorer, as it did not alert or block our services exploits, even when we applied Restrictive Policy.
The one success was that CA detected and blocked our zero-day exploit under default policy.
|Though McAfee's management GUI was disappointing, ePO's reporting features are excellent, including more than 70 different reports that break down all aspects of the enterprise. The point-and-click custom report creation tool is stellar, making it easy for people who are not database experts to massage the information into highly useful reports.
Symantec is also solid, offering more than 70 reports, with impressive performance. Symantec's custom reporting capabilities are focused on defining filters for its existing reports to create useful subsets, a valuable capability but somewhat less flexible than McAfee.
The IBM ISS reporting tool provided good coverage, addressing long-term trends and top attacked and infected machines. However, getting at the report files is a little obscure. Admins have to remember where they were generated in the file system to open the report from within the management GUI. Further, to open a report, you have to right-click on it and go to "Properties," a bizarre GUI twist that takes some getting used to.
Trend Micro's reporting is handled by a separate product, Trend Micro Control Manager, which is not tightly bundled into the existing management GUI, making a little more work for installation and use. On the positive side, this separate reporting tool applies to all Trend Micro enterprise products, including gateway security appliances, antispam products, etc. It's included in the purchase of the endpoint suite, and provides a full complement of well-laid-out reports.
eEye's built-in reporting features are decent and offer some features for creating custom queries in its published database schema. However, building custom or tweaked report queries is a complicated process, even using the built-in templates.
CA's reporting for antivirus and antispyware is stellar, with more than 70 reports available. Unfortunately, CA's HIPS and firewall features offer very little reporting, with only about a dozen high-level reports providing much less visibility into these important aspects.
Sophos' reporting capabilities are quite skimpy. Only about a dozen reports are available. They don't include Top 10 style reports of most infected systems, users or groups. The look and feel of the reporting engine makes the product appear better suited for small and medium businesses, rather than large enterprises. However, Sophos publishes its database schema for customers to use with third-party reporting tools, such as Crystal Reports.
|INTEGRATION OF COMPONENTS
Endpoint security suites should integrate disparate components into a coherent, manageable whole. Most of the vendors have worked hard to integrate various aspects of their solution, with high marks going to eEye, McAfee, Sophos and Symantec.
Trend Micro's antivirus/antispyware integration is decent, but integrating the personal firewall and HIPS, licensed from Third Brigade, needs some work. These components are a plug-in inside the management GUI, with a separate set of configuration screens that don't have the same look and feel of the configuration of antivirus and antispyware. Further, on the client side, antivirus and antispyware is a completely separate program from the HIPS software.
CA was never one for deep integration of components in its antimalware solutions. Its endpoint security product continues to separate management of antivirus and antispyware on different screens, but at least they both are available in one GUI application. CA's HIPS, on the other hand, is a separately purchased product. It is installed and managed using its own GUI and a separate client package is installed on protected workstations.
|In the End, Take It Slow
Symantec's new offering looks very solid, and eEye is a worthy new competitor in the endpoint security space. Trend Micro has a decent solution and a promising plug-in architecture for future expansion. CA and Sophos did reasonably well, but neither shined consistently. Finally, we were very disappointed with the numerous glitches, unfortunate design decisions and poor performance of McAfee and IBM ISS.
Regardless of which vendor you choose, keep in mind that the endpoint security market is relatively immature--witness our beta testing of three major vendors--and the complexity of any of these products warrants a carefully planned deployment strategy. We urge you to experiment with the products on your own laboratory test systems with images from your production environment to make sure they don't have any adverse consequences on your particular application mix.
Double check default policy settings to make sure they offer reasonable protection, and if not, adjust them for your environment and risk profile. And, finally, have your support staff become familiar with the various quirks of these management GUIs before production roll-out.