Security.com

What is shellcode and how is it used?

By Michael Cobb

Shellcode exploits have troubled security teams for decades. Learn what shellcode is, how it works, the main types of shellcode and how to protect against attacks that use shellcode.

What is shellcode?

Shellcode is part of the payload in the exploitation of a software vulnerability to take control of or exploit a compromised machine.

The word shellcode literally refers to code that starts a command shell -- an instance of a command-line interpreter, such as the shell /bin/sh on Linux or cmd.exe on Windows. The term now also embraces any bytecode that can be executed once the code is injected into a running application, even if it doesn't spawn a shell.

Common shellcode objectives include installing a rootkit or Trojan horse, stopping antimalware programs, obtaining sensitive data or downloading files to further compromise the targeted device.

How does a shellcode exploit work?

Shellcodes are injected into computer memory. After the exploit code causes what would normally be a critical error in the targeted program, the program jumps to the shellcode and is tricked into executing the attacker's commands -- all with the privileges of the process being exploited.

A shellcode exploit consists of two major components:

  1. The exploitation technique's objective is to insert the shellcode and divert the execution path of the vulnerable program to the shellcode so that it can run the code in the payload.
  2. The payload is the component that executes the attacker's malicious code.

For example, shellcode execution can be triggered by overwriting a stack return address with the address of the injected shellcode. As a result, instead of the subroutine returning to the caller, it returns to the shellcode and spawns a shell.

Writing a shellcode to use as a payload to execute something is the easier part of crafting a successful exploit. Attackers also need to find the address where the shellcode has been stored and gain control of the Extended Instruction Pointer register, which points to the next command, in order to run their exploits.

Types of shellcode exploits

Shellcode can be either local or remote:

Single buffers often have limited space where attackers can inject their entire payload into a remote target process. To overcome this restriction, hackers use various techniques, including the following:

The most common programming errors used to insert shellcode exploits are buffer overflows. Buffers are temporary areas for data storage. A buffer overflow occurs when more data is put into a buffer than it has capacity for. There are two main types of buffer overflows:

  1. A stack-based buffer overflow attack occurs when an application's stack -- the area that stores requests -- is exploited.
  2. A heap-based buffer overflow attack occurs when an application's memory space is exploited.

Shellcode may be used in other vulnerability exploits, including the following:

Why are shellcode exploits challenging?

NIST's Common Vulnerabilities and Exposures, Common Weakness Enumeration and SANS consistently rank buffer overflows as a top application vulnerability. The vulnerability is well known, but it continues to plague security teams. Also, with many attackers now using self-decrypting, polymorphic and various static but nonstandard encodings, intrusion detection systems cannot detect their shellcode using simple signature matching.

Anyone writing shellcode needs to have an in-depth understanding of assembly or machine code, C and C++ languages, processor architecture and the targeted OS. For example, Windows shellcode is quite different from Linux shellcode. Unlike Linux, Windows does not have a direct kernel interface. The addresses of the functions found in Windows' dynamic link libraries (DLLs) vary from version to version, while Linux has a fixed numbering system for all kernel-level actions.

The main reason shellcode exploits are possible is because the application or library doesn't correctly validate the data it is handling. OSes allocate specific and finite amounts of memory to hold data, such as variables, values and arrays. These storage areas, called buffers, are generally created at the time a program is loaded or dynamically during program execution. When data exceeding the buffer's capacity is input, it overflows the buffer, and the excess data spills into other memory areas or buffers, overwriting some or all of the contents held in that memory space.

Software developers need to properly inspect how much data is written into a specific part of a program's code. In higher-level languages, like Java and C#, such coding errors are harder to make. But, because there are so many applications written in lower-level languages, like C and C++, these exploits are likely to be around for some time to come.

How to protect against shellcode exploits

Security teams can find details about specific shellcode exploits in databases such as Exploit Database and 0day Today.

To protect against exploits that inject shellcode into vulnerable programs, enterprises need a multilayered security strategy. Hackers use automated scanners to look for applications using known vulnerable code, so enterprise firewalls and device controls need to be set to stop unwanted and known malicious connections. Monitoring controls also need to be in place to spot and quarantine unusual activity on the network using static and behavioral AI to stop the attack before it can do any lasting damage.

In addition, all applications need hardening to make them more resilient against exploits by ensuring that security controls, such as advanced memory protection and data execution prevention, are activated. In-house applications should be subject to static and dynamic tests, including direct attempts to inject shellcode into running processes using tools such as Metasploit and PowerSploit.

Finally, to check whether these combined defenses are effective, administrators can use penetration tools to test whether they detect and, most importantly, prevent exploits attempting to run shellcode.

Shellcode exploit examples

A classic attack using shellcode is the exploitation of the JpegOfDeath vulnerability in gdiplus.dll, which intentionally causes a buffer overflow condition. Anyone who opens a JPEG image created with JpegOfDeath exploit code invokes a buffer overflow, which takes advantage of this condition to inject shellcode into memory that is executed when the overflow occurs.

A watering hole attack on a North Korea-related news site used JavaScript to exploit a chain of Google Chrome and Microsoft Windows vulnerabilities (CVE-2019-13720). It injected shellcode, along with a Portable Executable -- Windows executable and object files -- for the shellcode to execute, into the memory of an audio buffer of a WebAudio component.

The 2021 Microsoft Office MSHTML Remote Code Execution Vulnerability (CVE-2021-40444) enabled remotely hosted shellcode to be loaded into the Microsoft Address Import Tool and be executed without user interaction.

The Purple Fox exploit kit exploited a memory corruption vulnerability in Internet Explorer (CVE-2021-26411). It injected shellcode that runs a PowerShell statement to download images from a remote server and execute further PowerShell scripts extracted from the images.

Even though the threat from shellcode isn't new, this article provides essential background on what shellcode exploits are, how they are launched and how to defend against local or remote shellcode attacks. Understanding how shellcode can exploit a compromised application or machine can help organizations of all sizes know what to look for and protect themselves.

25 Jan 2022

All Rights Reserved, Copyright 2000 - 2024, TechTarget | Read our Privacy Statement