Memory Corruption, All the Way Down to the Register

A CPU does not know what your program meant. It has one register that holds the address of the next instruction, and memory corruption is the art of deciding what goes into it. This is the first-principles account, down to the stack frame, the heap chunk, the vtable pointer, and the instruction pointer: why memory corruption is possible at all, the four ways a program's model of memory diverges from the machine's, why corrupted data becomes controlled execution, and why every mitigation we have is one move in an arms race over a fixed board.

A CPU does not know what your program meant. It knows one thing well. It has a register, the instruction pointer, that holds the address of the next instruction. On x86-64 it is called RIP. The machine fetches the instruction at RIP, decodes it, runs it, advances RIP, and repeats. That loop is the whole computer.

Memory corruption is the art of deciding what goes into that register. Everything else, every overflow and every freed pointer, is plumbing to reach it. If you understand the register, you understand why this entire class of bug exists and why it has been the most valuable one for thirty years.

Why it is even possible

Two facts from the ground floor, and the rest follows.

Code and data share one space. This is the von Neumann design, and it is not an accident you can patch. The stack that holds your local char buf[64] also holds the return address that will become RIP when the function finishes. They sit inches apart in the same flat array of bytes. A C++ object that holds your data also holds, in its first eight bytes, a pointer the CPU will call through. Control and data are not in separate vaults. They are interleaved, everywhere.

A pointer is an integer, and nobody checks it. In C, arr[i] means *(arr + i). The machine takes the base address, adds i times the element size, and reads or writes there. Nothing in the hardware asks whether i is still inside arr. The bounds of the object existed in two places only: the programmer’s head and the language’s rules. C enforces neither at runtime. So a write “outside the lines” is not a fault. It is an ordinary write to whatever address the arithmetic produced, including an address that holds control.

Put those together. The machine will happily let an index walk off the end of a buffer and keep writing, and a few bytes past the end of that buffer is a value the CPU will later load into RIP. That is the entire opening.

Here is the essence, and everything below is detail hung on it. Your program thinks in typed objects with sizes and lifetimes. The machine has none of that. It has a flat array of bytes and a register that reads the next address out of it. Memory corruption is what happens when the machine’s reality, no types, no bounds, no lifetimes, is made to punish the program’s abstraction. Every variety that follows, on the stack, on the heap, through a bad cast or a wrapped integer, is one way to pry open that single gap between what the program believes and what the silicon actually does.

The stack, concretely

Here is what a function call actually does on x86-64. call foo pushes the return address, the address of the instruction right after the call, onto the stack, then jumps to foo. Inside foo, the epilogue runs ret, which pops the top of the stack back into RIP. The return address is data on the stack that becomes the program counter. Hold that sentence.

The stack grows downward, toward lower addresses. A local buffer is allocated at a lower address than the saved return address, so writing past the end of the buffer moves up, toward the return address. The frame looks like this:

higher addresses
    ...                 caller's frame
    [ return address ]  <- ret loads this into RIP
    [ saved RBP      ]
    [ buf[63]        ]  <- writing past here goes UP, toward the return address
    [ ...            ]
    [ buf[0]         ]  <- a copy that does not check length starts here
lower addresses

Now give the function a copy that trusts an attacker-controlled length, or an old call like gets that has no length at all. The write starts at buf[0] and runs upward. Past buf[63] it crosses the saved frame pointer. A few bytes further it lands on the return address and overwrites it with attacker bytes. The function finishes. ret executes. The CPU loads those bytes into RIP and fetches the next instruction from an address the attacker chose.

That is the classic stack smash, and the reason it has a special place in this field is in the last sentence. Nothing exotic happened. A loop copied too many bytes into an array, and because control lives next to data, the program counter became attacker data.

The heap, concretely

The stack is the clean case. Most modern corruption is on the heap, where the control data is subtler and the mechanism is more interesting.

The heap is a pool the allocator hands out in chunks. free(p) returns a chunk to a free list, but p still holds the old address. It is now a dangling pointer, aimed at memory the allocator considers available. The allocator will hand that same chunk to the next request of a similar size.

Use after free is the program continuing to use p after the free. On its own that is a latent bug. It becomes an exploit when the attacker arranges for a new object to be allocated into the freed chunk between the free and the next use. Now p, which the code still treats as the old object, actually points at memory the attacker just filled. If the old object was something the program calls through, the attacker has reached control.

Which brings in the sharpest case, the C++ vtable. A polymorphic object stores, in its first field, a pointer to its vtable, an array of function pointers. A virtual call compiles to roughly this:

mov   rax, [rcx]        ; load the vtable pointer from the object
call  [rax + 0x10]      ; call the method at a fixed slot in the vtable

The CPU loads a pointer out of the object and calls through it. If the attacker controls the bytes of that object, through a use-after-free or a type confusion, they control the vtable pointer. They point it at memory they filled, put an address of their choosing in the called slot, and the indirect call goes there. Same ending as the stack smash, reached through an object field instead of a return address: a value in writable memory became the program counter.

Type confusion is the other road to the same place. The language believes a region is type A; it is really type B. A field that is an integer in B is a pointer that gets dereferenced in A. The label lied, and a value the attacker controls as data in one view becomes an address the CPU follows in the other.

Integer overflow, the quiet root

One more, because it is the cause behind a great many spatial bugs and it hides in plain arithmetic. Allocation sizes are computed, usually as count * size, in a fixed-width register. If that product exceeds what the register holds, it wraps. malloc is handed a small number and returns a small buffer. The code, still believing it asked for the big size, writes count elements into it and runs off the end. No pointer was misused. A multiply overflowed a 64-bit register and the math told a lie, and a spatial overflow fell out of it.

Why corruption becomes control

Step back and the whole class is three primitives the attacker is climbing toward.

• A controlled write: put a chosen value at a chosen address.
• A controlled read: learn the contents of a chosen address.
• A controlled jump: put a chosen value into RIP.

They convert into each other because control data is scattered through writable memory. Return addresses and saved frame pointers on the stack. Function pointers and vtable pointers in objects. Entries in the table of resolved library functions. The allocator’s own metadata in the heap. Length fields and permission flags that gate later decisions. Corrupt the right one and a write primitive becomes a jump primitive. This is why boundary seven is the crown of the taxonomy. The other six boundaries get you data or a borrowed action. This one reaches the register that decides what the machine does next.

The arms race, at the level of the machine

Every mitigation in memory safety is a single, specific move against one of the facts above. Read them as an escalation, because that is what they are.

The stack canary. The compiler places a random word between the local variables and the saved return address, and the function epilogue checks that the word is unchanged before it runs ret. A linear overflow that wants the return address has to cross the canary, and the mismatch aborts the program. It removes the easy stack smash. It does not help when the overflow is not linear, or when the attacker can read the canary first and write it back.

Non-executable memory. The memory management unit marks the stack and the heap as no-execute. If a buffer holds attacker bytes, the CPU refuses to fetch instructions from that page. This killed the original move, jump to your shellcode sitting in the buffer. The answer was to stop injecting code at all. Reuse code that is already there and already executable. Return-to-libc jumps into a standard library function. Return-oriented programming generalizes it: find short sequences of existing instructions that each end in ret, called gadgets, and chain them by writing a series of their addresses onto the stack, so each ret walks to the next. The overflow never introduces code. It only supplies addresses of code that was already present. No-execute memory is intact and irrelevant, because nothing new was executed.

Address space layout randomization. If the attacker needs the address of a gadget or a buffer, take it away. Randomize the base of the stack, the heap, the libraries, and the executable on every run. Now the chain cannot be written, because the attacker does not know where anything is. This is why information leaks became the center of gravity. A single leaked pointer, pulled through a controlled read, de-randomizes an entire region and hands back every address derived from it. Randomization turned exploitation into a two-step game: leak to learn the layout, then corrupt.

Control-flow integrity and shadow stacks. The frontier. Constrain where indirect calls, jumps, and returns are allowed to land, to a set of legitimate targets the compiler knew about, so even a controlled pointer cannot send execution to the middle of a gadget. A shadow stack keeps a separate, protected copy of return addresses and compares against it before ret, so a clobbered return address is caught.

Notice the shape. Canary, no-execute, randomization, control-flow integrity. Each one removes exactly one assumption the attacker was standing on, and each time the attacker finds the next assumption still holding. The board does not change. Control data still lives in writable memory. The game gets more expensive, slower, and narrower, and it does not end. It is the same lesson the offense-defense contest teaches everywhere. Both sides level up, and the fight speeds up rather than finishing.

Why the whole board vanishes in managed languages

Now the contrast that pays off the taxonomy. In Go, Java, Python, and the safe part of Rust, this entire article is a story about a machine you never touch.

• Every array access is bounds-checked. The index is compared against the length, and an out-of-range access throws instead of writing. The index cannot walk off the object onto a return address, because the walk is stopped at the edge.
• Lifetimes are managed. A garbage collector or an ownership system makes sure memory is not reused while a pointer to it is live, so there is no dangling pointer and no use-after-free. Rust proves it at compile time with ownership and borrowing. The others enforce it at runtime with a collector.
• Memory cannot be reinterpreted as the wrong type. The runtime does not let you treat an integer as a pointer and follow it.

The four divergences, spatial, temporal, type, and integer-into-allocation, simply cannot happen. You pay for it, and the price is real: a comparison on every access, a collector that pauses or a borrow checker that argues with you. In exchange, boundary seven is gone. Memory safety does not lower the total amount of risk in a system. It relocates the fight. The bug moves off the register and into the logic, the authorization, and the injection of boundaries one through six. That is the deep reason the two battlefields in the companion piece need different weapons. One of them fenced off the register.

Why this still matters, more than before

It would be easy to conclude the register is a solved problem, now that the world writes its web code in memory-safe languages and a model autocompletes the rest. It is the opposite.

The model writes your handler, and the managed runtime catches its off-by-one. But the engine running the model is C++. The tensor kernels under it are C and assembly, written for speed, with the checks left out on purpose. The image, video, font, and document parsers at the bottom of every stack are C, and they eat untrusted bytes from the network all day. None of that gets safer because the layer above it got smarter. The one place a single out-of-bounds write still lands in RIP is exactly the place no copilot and no garbage collector reaches.

Understanding that layer, down to the stack frame and the vtable and the register, is not nostalgia. It is the part of security that does not commoditize, because it was never about typing the exploit. It was about knowing, precisely, how the machine can be made to do the wrong thing, and therefore how to stop it.

I occasionally advise small teams on backend reliability, Go performance, and production AI systems. Learn more: /services