As security researchers become more adept at detecting various forms of malware, attackers are beginning to develop techniques to prevent the detection and analysis of their creations.
In the past, the hackers who wrote viruses and worms spent little or no time trying to make their malware stealthy. They were far more concerned with finding effective techniques for spreading their viruses and causing the maximum amount of damage than with preventing security managers from finding the viruses. They figured, and rightly so, that by the time anyone found the virus or worm, it would have done its damage.
But as the focus of malware authors has shifted from digital vandalism to penetrating high-value networks for profit, evading detection has become a top priority. Their goal now is to drop a rootkit, bot or other piece of malware onto a target machine without being noticed and have the program stay in place for weeks or months while it gathers passwords, customer records or other valuable data, said Lenny Zeltser, a speaker at the Information Security Decisions conference here this week. Zeltser, the information security practice leader at Gemini Systems in New York, and an instructor at The SANS Institute, highlighted a number of techniques that are gaining favor in the hacker community at the moment.
Two such techniques in particular that have been seen in the wild of late are virtual machine detection and debugger detection. Antivirus researchers and security specialists in large enterprises commonly deploy virtual machines (VM) on their research networks and then infect those VMs with samples of new malware. This enables them to observe the behavior of the malware and analyze it without having to infect the underlying operating system on the PC or server. Virus writers of course know this, and many of them have begun including code in their viruses that can check to see whether it is running on a VM and then shut down if it is not running on the machine's actual operating system. This technique won't prevent researchers from eventually analyzing the program, but it can slow down the process, thereby delaying the creation of a signature for the malware.
Zeltser said he has seen examples of other pieces of malicious code that check to see whether they are attached to a debugger, another tool that researchers use to analyze viruses and worms. One way virus writers accomplish this is to have their programs time how long it takes for the code to execute, and if it's taking too long—an indication that a debugger is attached—the program shuts down.
"This is a very clever idea, and it's one that I've seen a couple of times and I think we'll probably see more in the future," Zeltser said.
Another technique, which at this point is believed to be only theoretical, is using a VM-based rootkit to compromise the host operating system and then implement malicious processes on the machine. To do this, the malware installs a virtual machine monitor underneath the host operating systems, then hoists them onto the virtual machine. The rootkit is then essentially undetectable because the host operating system can't access them. A group of researchers from Microsoft Corp. and the University of Michigan published a paper earlier this year on this technique and the proof-of-concept rootkit, called SubVirt .
"It's a very elegant idea that's likely to gain traction in the near future," Zeltser said. "The victim machine doesn't control the real OS, so all of their processes are executing within the virtual machine. Very difficult to detect and remove."