Part 6 of 7 · 3 min read
The stack protector
The stack canary
In plain words: A different kind of overflow happens inside the stack. A local array of 8 bytes that receives 32 bytes overwrites the data above it in the frame, including the saved return address. A program that returns through a corrupted address may run attacker code. Compilers add a defence: a stack canary, a secret value placed in the frame between the local arrays and the return address. The function checks the value before returning. If the value has changed, a buffer overflow has happened, and the program stops at once instead of returning through a bad address.
The small test program copies its argument into char buf[8] with strcpy:
| Build | Argument of 32 A characters |
|---|---|
With the stack protector (the Ubuntu default, -fstack-protector-strong) |
prints *** stack smashing detected ***: terminated, exit code 134 |
With -fno-stack-protector |
segmentation fault, exit code 139 |
The protector turns an unpredictable crash, or worse a silent takeover, into a clear stop. It does not remove the bug. It costs a few instructions in functions that have local arrays. The real fix is to check the length (snprintf, strncpy with care, or memcpy with a checked size). Level 10 returns to this defence with others, such as non-executable stacks and address randomisation.
Stack canary
A stack canary is a value that the compiler places in a stack frame between local buffers and the saved return address, and checks before the function returns, so that an overflow of a local buffer is detected and the program is stopped.
Example: when 32 bytes are copied into an 8-byte local buffer, the canary changes, and the function ends with "stack smashing detected".
Interview angle
"What happens when a recursive function has no base case?" A good answer: each call adds a frame, the stack grows page by page through page faults, the limit (8 MiB by default) is reached after tens of thousands of calls, and the kernel sends SIGSEGV. It mentions that a big local array fails at once, and that the cure is a base case, a loop or the heap.
Check yourself: 1. Where in the frame is the canary placed? 2. What happened without the stack protector in the test? 3. Does the canary fix the overflow bug?
Words you will meet
| Word | Meaning |
|---|---|
| Stack | The region used for calls, growing toward lower addresses |
| Stack frame | The block one call uses on the stack |
| Stack pointer | The register that marks the top of the stack |
| Return address | Where execution continues after a call |
| Stack limit | The largest size the stack may reach (ulimit -s) |
| Stack overflow | The stack passes its limit |
| Guard page | A page with no permissions at the end of a stack |
| Stack canary | A check value between buffers and the return address |
RLIMIT_STACK |
The resource limit that sets the stack limit |
| Alternate signal stack | A separate stack on which a signal handler runs |
Quick notes
- A stack frame is the block of stack memory that one function call uses, holding its return address, saved registers and local variables, which is created when the function is called and dropped when it returns.
- The stack size limit is the largest size to which the kernel will grow the stack of a process, held in the resource limit
RLIMIT_STACK(8 MiB by default on most Linux systems, shown in KiB byulimit -s). - A stack overflow is the failure that occurs when a process needs more stack than its limit allows, usually through a recursion that is too deep or a large local.
- A guard page is a page with no access permission placed next to a stack (or another buffer) so that any access that runs over the end faults at once instead of silently damaging the memory beyond.
- A stack canary is a value that the compiler places in a stack frame between local buffers and the saved return address, and checks before the function returns, so that.