 |
| Listen to Noah Schiffman's tip |
| Download Schiffman's RE:Trace framework tip to your PC or favorite mobile device. |
|
|
|
|
Reverse engineering is regarded as one of the most difficult specialties in the hacker community. The deconstruction and analysis of software and systems to understand their inner workings is a complex task. It requires a thorough understanding of kernel functions, knowledge of machine code and experience using disassemblers. While a number of reverse engineering tools exist, a modification of tracing tool DTrace has led to the creation of the robust reverse engineering framework known as RE:Trace.
DTrace: Easier application analysis
This dynamic tracing tool was introduced with the release of Sun Microsystems Inc.'s Solaris 10 operating system in 2005. This low-level debugging and monitoring framework allows detailed analysis of system resources used by applications. Among its many advances in kernel debugging, it greatly reduces the time needed for process activity and performance analysis. It has the unique ability to dynamically examine processes across the entire software stack -- from kernel mode through userland. DTrace provides comprehensive system-wide information, while allowing the user to receive customized details with granular control.
DTrace receives its shell script interface from the block-based D programming language. Its absence of loops and pointers augment system performance, making it safe for use on production systems. The shared library, libdtrace, provides DTrace with a portal from user to kernel mode. The inspection of kernel resource utilization is queried through probes. Defined in the D language script, kernel interrogation is precisely defined by the probes' quartet of structured syntax: provider, module, function and name.
RE:Trace: Engineering the tool
In order to create the RE:Trace platform, the data limitations of DTrace are overcome by its integration with the Ruby programming language. Furthermore, this object-oriented wrapper extends compatibility to other security applications, such as Metasploit. However, of greater use to the reverse engineer is its ability to work with the Interactive Disassembler, or IDA. Their compatibility is brokered by the IDA plugin, idaRub. Introduced in 2006, idaRub wraps the IDA SDK for access from Ruby, affording the rapid development of advanced IDA plugins for Ruby environments.
The combination of DTrace and Ruby provides the starting point for reverse engineering. Although, RE:Trace utilizes this pair functionally, its real strength lies in its prepackaging of reverse engineering tasks and processes. The detection and instruction pinpointing of stack-based buffer overflows, heap overflow monitoring and block-level tracing with code coverage visualization are bundled into the RE:Trace framework. The incorporation of these features, in addition to others, has created this unique and powerful reverse engineering framework.
RE:Trace: How to use it
Reverse engineering malware is an important aspect of security research, which studies the behavior of malicious code using several disk, network, registry and process monitoring tools. RE:Trace provides an effective combination of debugging and disassembly for reverse engineering malicious executables. Its ability to control code execution while simultaneously performing process inspection make it an excellent tool for dynamic analysis.
Modern malware employs a number of mechanisms to thwart its discovery and deconstruction. Common malware tactics include embedding encrypted strings that are decrypted at the stack, and sending kill commands to antivirus and system-protection processes. Furthermore, numerous platform specific anti-debugging methods have been in use for long time.
However, RE:Trace can evade many of these measures through the strategic placement of probes. Using improved EIP register monitoring, instructions responsible for stack overflows can be precisely identified. As dynamically allocated memory has become a greater attack vector, the need for heap monitoring has become important. Addressing this growing threat, RE:Trace not only monitors dynamic allocation functions, but is also able to hook functions aimed at heap overflows.
Summary
The potential of RE:Trace has yet to be realized. Its versatility allows the creation of rootkits and malware, as well as their disassembly and analysis. Hopefully it will continue to be used as a system analysis tool for developers and administrators. However, since resources are almost always misused for malicious intent, its application in security may prove to be its greatest value.
About the author:
Noah Schiffman is a reformed former black-hat hacker who has spent nearly a quarter century penetrating the defenses of Fortune 500 companies. Today he works as an independent IT security consultant specializing in risk assessment, pen testing, cryptography and digital forensics, predictive analysis models, security metrics and corporate security policy. He holds degrees in psychology and mechanical engineering, as well as a doctorate in medicine from the Medical University of South Carolina. Schiffman is based in Charleston, S.C.
