What are the defences against stack buffer overflow?

The stack buffer overflow occurs when a variable value exceeds its allocated space and overwrites code placed on the stack. The overwriting text is written so that the executing program considers it a part of the code. The program tries to execute this text, which ultimately crashes the application or does something harmful.

Note: Stack buffer overflow is also known as "Stack Smashing."

Prevention

The following are the defense mechanisms in place for this kind of vulnerability:

  • Stack canaries
  • Non-executable stack
  • Address space layout randomization

Each one is discussed in detail below:

Stack canaries

Stack canaries are snippets of randomly generated numbers placed inside the stack after the allocated space for the user input. This helps while checking if the stack has been smashed. If so, the canary will have changed.

Consider the following code snippet. It is vulnerable to a buffer overflow attack.

void foo (int a, int b) {
char buf[16];
gets(buf);
return;
}

This code is visualized below on the stack. We are assuming that one address space is four bits.

%0 node_1659596287707 6 node_1659596287400 3 node_1 ret addr; node_1659596346559 file pointer node_1658230910015 0x0F0F0F node_3 buf[12]-buf[15] node_1658229727340 buf[8]-buf[11] node_1658229759483 buf[4]-buf[7] node_1658229791961 buf[0]-buf[3] node_1659595835757 ... node_1658229808404 some code top_node top node_1658229808404->top_node

Here, if the user's input is longer than the allocated amount, it will overwrite the return address. Before that the stack canary (0x0F0F0F) will be overwritten. The code before execution will check if the canary has been compromised and will raise an exception if it is the case.

Let's consider the following user input: "good morning, you are hacked." The stack visualized above would then become:

%0 node_1659596155990 ked/0 node_1659596129291 hac node_1659596108221 are node_1659596196491 you node_1659596153868 ning node_1659596202991 mor node_1 good node_2 ... node_3 some code top_node top node_3->top_node

The return address has been overwritten here by the string "ked/0." Since the canary was also overwritten, the execution of this code will be halted.

Address space layout randomization

This technique randomizes the layout of the stack to prevent the attacker from knowing where the overwritten return statement will return the code execution.

Note: The main goal of overwriting a return statement is to direct the instruction pointer to another executable code snippet inside the stack. An example is that of the "return-to-libcA return-based buffer overflow attack that utilizes the methods present in the libc library to execute malicious code." attack.

Linux-based systems use ASLRAcronym for address space layout randomization. to randomize the starting addresses of the heap and stack, making it hard for the attacker to guess where the desired piece of instruction is stored on the stack. We can use the following command in the terminal to enable or disable ASLR on a Linux-based system:

$ sudo sysctl -w kernel.randomize_va_space=0

Setting the above statement to 0 disables randomization.

Non-executable stack

As the name suggests, this technique prevents the execution of any statement placed inside the stack. Some Linux–based systems used to allow executable stacks, but now they are disabled by default. Binary images and shared libraries of programs had to declare whether they required a non-executable stack or not. This could be set with a flag while compiling the object file of the code.

$ gcc -z execstack -o test test.c

or

$ gcc -z noexecstack -o test test.c

The code above demonstrates setting the stack executable or nonexecutable while compiling a C file.

Free Resources

Copyright ©2025 Educative, Inc. All rights reserved