Signature-based Detection
To test signature-based detection, we assembled 54 known spyware components (47 .exe files, four DLLs and three JavaScripts). We tested detection of the products in three stages:
- We attempted to copy our spyware to a test machine to see if the product had real-time protection to prevent potential malware from being written to the file system.
- We disabled the product, copied the spyware onto the target file system, re-enabled the product and performed an on-demand scan.
- We copied the spyware onto a machine with the product disabled, re-enabled the product and attempted to launch each of the 47 executables to see if real-time protection followed by an on-demand scan would thwart the malware.
In analyzing our results, we looked at the overall effectiveness of each product. We awarded grades based on their ability to block spyware at each stage, placing the greatest weight on their effectiveness in stopping spyware before it had a chance to run on the system.
The clear winner in this category was McAfee, which kept us from copying 37 of the 47 executables. It found 25 of the 47 during our on-demand scan, and it left only four processes running at the end of our test series. This strong showing, which detected more than three times the number of malicious programs posted by its nearest competitor, is undoubtedly the result of its multifaceted approach to detection.
While they weren't as comprehensive as McAfee, both CA and Trend Micro performed quite well overall, each detecting 12 of the programs in the on-demand scan. Like McAfee, CA left only four processes running at the end of the testing. However, CA demonstrated no ability to block spyware from being copied to our machine.
Trend Micro, on the other hand, detected and blocked 12 of the executables we attempted to copy to the machine. But, it left a few more running processes at the end.
Overall, we felt that these results balanced off, earning CA and Trend solid "Bs".
Webroot's detection rate, finding 10 of 47 specimens during the on-demand scan, was only slightly lower than CA or Trend Micro. However, the product offers no provision for blocking spyware from being copied to a computer. It left five running processes after the testing.
eSoft's decision to leave its Active Defense Shield off by default dropped it to the middle of the pack. When we enabled this shield, eSoft was able to block seven executables from being copied to the computer, matching the number it found during the on-demand scan. It also posted an impressive overall performance, leaving only four running processes when the tests were completed.
Lavasoft's lack of any real-time detection hurt its score, as it failed to block our attempts to copy spyware files to our test machine. In addition, it identified only eight of 47 executables during the on-demand scan. In the end, it left nine malicious programs running.
Finally, while SurfControl blocked six files during our attempts to copy spyware to our test machine, that's about as far as it got. It managed to detect those same six files during on-demand scanning, but failed to block or clean anything new. It left seven malicious files running.
