Problem solve Get help with specific problems with your technologies, process and projects.

How to stop buffer-overflow attacks and find flaws, vulnerabilities

In this tip, which is part of our Web Application Attack Security Guide, learn how to stop buffer-overflow attacks from infiltrating your systems and learn how to find buffer-overflow flaws and vulnerabilities with protection and defense methods and best practices.

Buffer-overflow flaws and vulnerabilities can cause serious damage to an organization by means of an embarrassing...

and expensive data security breach or by enabling an attacker to assault a Web application and gain control of a corporate machine.

This tip delves into the inner workings of a buffer-overflow attack to uncover how they unfold and identify ways in which to stop a buffer-overflow attack from compromising enterprise Web applications.

What is a buffer overflow and how does it work?

A buffer is a temporary area for data storage. When more data than was originally allocated to be stored in a buffer gets placed there by a program or system process, the extra data will overflow, hence the name, causing some of that data to leak out into other buffers, which can corrupt or overwrite whatever data they were holding.

In a buffer-overflow attack, the extra data sometimes holds specific instructions for actions intended by a hacker or malicious user; for example, the data could trigger a response that damages files, changes data or unveils private information.

Hackers would use a buffer-overflow exploit to take advantage of a program that is waiting on a user's input. There are two types of buffer overflows: stack-based and heap-based. Heap-based, which are difficult to execute and the least common of the two, attack an application by flooding the memory space reserved for a program. Stack-based buffer overflows, which are more common among hackers, exploit applications and programs by using what is known as a stack: memory space used to store user input.

A stack can only hold a certain amount of data, and if the input string is longer than the amount of space reserved, then the result is an overflow, creating a security hole. Savvy malicious hackers seek out these flaws with specially written commands that cause an overflow and trigger an attack. Once the malicious command causes the overflow, the hacker must still execute the command by indicating a return address that points to the command. The buffer overflow causes the application to crash partially, but it tries to recover by going to the return address, which has been redirected to the malicious command by the hacker.

When the buffer-overflow attack runs the command found at the new return address, the program thinks it is still running. This means that the command prompt window that has been opened is running with the program's or application's same set of executable permissions as the application that was compromised, allowing the hacker to gain full control of the operating system.

More on this topic

  • Web applications are the conduit for buffer-overflow attacks on the Web server. These buffer-overflow tools can help thwart attacks.
  • In this two-part series, Brien Posey explains how to identify and prevent buffer-overflow attacks on home-grown applications.

How to stop a buffer overflow from attacking applications

Now that you know how a buffer-overflow attack works, you can better grasp how to prevent them from infiltrating your system and taking control of your applications. Here are some ways to bulk up your defenses and prevent a buffer overflow:

1. Avoid using library files: Library files, which are used in programming language and are inherently insecure, are a target for hackers during application attacks. Any weakness found by a hacker in a library file will also exist in all applications that use that library file, giving hackers a glaring target for a potential attack.

2. Filter user input: Filtering out possibly dangerous HTML code and characters that could cause database problems. For example, in ASP code, the apostrophe, quotation mark and ampersand symbols are all reserved symbols. These reserved symbols can't be included within a user's input or they will cause the application to crash. Filter them out and replace them with something else to avoid complications and problems.

3. Test applications: Be sure to test all applications prior to deployment; trying to break into every application to ensure secure coding. If the application breaks, it will be clear that there is a problem that needs to be fixed before a hacker is able to exploit it.


  Introduction: Web application security
  How to stop buffer-overflow attacks
  Prevent cross-site scripting hacks 
  Stopping SQL injection hack attacks
  Distributed denial-of-service protection

This was last published in January 2010

Dig Deeper on Application attacks (buffer overflows, cross-site scripting)

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.