TLB Miss, Cache Miss and False Sharing: Three Stalls Your Profiler Won't Name

What a TLB miss, a cache miss and false sharing each cost the CPU, why top still reports 100% busy, and how to see them with perf stat and perf c2c. Two threads doing identical work ran in 841 ms or 8118 ms depending only on where their data sat in memory.

top says both threads are at 100% CPU. Your profiler says the hot function is a simple counter increment. Everything looks busy and nothing looks wrong — and the program is still half as fast as it should be. The reason isn’t in your code; it’s in where your data lives relative to the cache. Three effects do most of this damage: cache misses, TLB misses, and false sharing. None of them show up as a function name in a flame graph.

The headline number for the worst of them, measured on one machine with CoreTracer: two threads doing the same increment loop finish in 841 ms or 8,118 ms depending only on how two integers are laid out in a struct.

Cache miss — the stall you can’t see

The CPU is fast; memory is not. An L1 hit is a few cycles; a miss that goes to L2, L3, and finally DRAM costs on the order of hundreds of cycles. During that time the core stalls — it retires no instructions, yet top still counts it as 100% busy, because “busy” to the OS means “not idle,” not “doing useful work.”

This is why a profiler can mislead. It tells you which function ran hot; it rarely tells you the function was hot because every iteration missed cache and sat waiting on DRAM. A hash lookup where each probe pulls a cold bucket, a linked-list walk with no spatial locality, a lookup table bigger than L2 — all of these read as “CPU-bound” while actually being memory-latency-bound. The fix is never “optimize the function”; it’s “change the access pattern so the data is there when you reach for it.”

CoreTracer has the minimal version of this (the sequential-vs-random demo in c_demo/cache_tlb_latency/): a 1 GB array of ints, summed once in index order and once through a shuffled index. Same loop, same 256 million loads, same instructions. The only thing that changes is the order the array is visited. On the same Xeon D-1521:

4.5x slower for reading exactly the same bytes. In the sequential pass the prefetcher has the next line in L1 before the loop asks for it; in the random pass nearly every load is a trip to DRAM, and the core spends the difference stalled, at 100% CPU the whole time. The gap is 4.5x rather than 50x only because these loads are independent and the out-of-order core keeps a dozen of them in flight; a pointer chase, where each load depends on the previous one, loses that overlap and gets far worse. This run recorded wall time only; the perf stat line further down is how you confirm the miss counts behind it.

TLB miss — the tax on every address

Before the CPU can even fetch your data, it has to translate the virtual address to a physical one, and it caches those translations in the TLB. Miss the TLB and the hardware walks the page tables — several dependent memory accesses of its own, often costing more than the cache miss that might follow.

TLB misses scale with how scattered your memory is and how many mappings are live. The place this bites hardest in production is dense multi-tenant hosts: many small processes or containers, each with its own page tables, thrashing a shared TLB. An inference server packing many tenants onto one box pays this quietly on every access, and no application-level metric attributes it. Huge pages exist precisely to shrink this tax — one TLB entry covering 2 MB instead of 4 KB — which is why they matter for large working sets.

CoreTracer measures this directly (c_demo/cache_tlb_latency/tlb_miss_demo.c, run through tlb_bench.sh). It maps 1 GB, touches one byte on every 16th page, and sweeps that 100,000 times, once with 4 KB pages (MADV_NOHUGEPAGE) and once with 2 MB transparent huge pages (MADV_HUGEPAGE). Same memory, same loop, same loads. perf stat on a Xeon D-1521:

Two out of five loads missed the TLB on 4 KB pages. On 2 MB pages the misses all but vanish and the same loop runs 1.7x faster with nothing else changed. The cache-misses row is the one to stare at: this test touches one byte per page, so every data access is a cache miss by design in both columns. The extra 2.9 billion misses in the 4 KB column are almost entirely the page walks themselves, the CPU reading page-table entries from DRAM before it can read your byte. A TLB miss is not one stall. It is several dependent memory reads, which is why it often costs more than the cache miss that follows. (Counters were multiplexed, so absolute counts are perf’s scaled estimates; the ratios are the point.)

False sharing — the one that punishes concurrency

This is the sharpest of the three, because it turns adding threads into a slowdown. Coherence hardware tracks ownership at cache-line granularity — 64 bytes on x86 — not per variable. So two threads writing two different variables that happen to live in the same line will fight over that line as if they shared it.

Here is the setup from CoreTracer, reduced to the part that matters:

#define CACHE_LINE_SIZE 64

// a and b land in the same 64-byte cache line
typedef struct { volatile int a; volatile int b; } shared_false_t;

// padding pushes b onto the next line
typedef struct {
    volatile int a;
    char padding[CACHE_LINE_SIZE - sizeof(int)];
    volatile int b;
} padded_t;

Two threads, pinned to separate physical cores, each hammering its own field:

false_sharing_thread1: bind_thread_to_core(0);  for (i…) { s->a++; __sync_synchronize(); }
false_sharing_thread2: bind_thread_to_core(1);  for (i…) { s->b++; __sync_synchronize(); }

Thread 1 only ever touches a, thread 2 only ever touches b. Logically independent. But with shared_false_t, a and b are in the same line, so every write by core 0 invalidates core 1’s copy of the whole line and vice versa. The line ping-pongs across the interconnect on every iteration. Swap in padded_t and the two fields sit on different lines — the contention disappears and nothing else changes.

The numbers from that run, same work throughout:

Same instructions, same iteration count, both cores at 100% in every run. The only variable is layout: a struct-field order in the first two rows, and in the third, two threads deliberately fighting over a single variable — the pure-contention ceiling. Between “padded” and “ping-pong” there is nearly a 10× difference that no CPU-utilization graph will ever explain.

How to actually see it

top and application profilers can’t distinguish useful cycles from stall cycles. perf can:

perf stat -e cycles,instructions,cache-misses,L1-dcache-load-misses,dTLB-load-misses ./bench
perf c2c record ./bench   # then: perf c2c report  — points at the exact false-shared line

Watch IPC (instructions per cycle) collapse and cache-misses climb between the packed and padded runs, and perf c2c will name the cache line two cores are fighting over. That’s the difference between “CPU is high” and “high doing what.”

Why this matters if you write services, not benchmarks

These three are the mechanism behind “I added cores and it got slower” and “the profile looks flat but latency is bad”:

• Cache misses on shared lookup structures — routing tables, feature stores, inference KV caches — where each access pulls a cold line.
• TLB misses on multi-tenant hosts with high page-table pressure; the denser the packing, the worse.
• False sharing in exactly the place teams add it by accident: per-thread counters, metrics, and sharded state packed tightly into one struct “to be cache-friendly,” which does the opposite.

The unifying lesson is that memory layout is performance, not an optimization pass you do later. A field reorder can be a 2× win; 64 bytes of padding can be the difference between concurrency that scales and concurrency that ships as a serial bottleneck. For AI infrastructure the stakes compound: inference servers run multi-tenant on cores you don’t choose, and all three effects get worse exactly when the box is busiest. The benchmarks are in CoreTracer — clone it, run perf, and watch the numbers move.

Related

• Video: Cache Miss, TLB Miss & False Sharing
• GitHub: CoreTracer — the false-sharing / cache / TLB benchmarks above, plus lock-free and memory-ordering experiments.
• Companion: Store→Load Reordering — x86 vs ARM64, the other place cache coherence and ordering surface.

Related reading: Rust vs C at the assembly level

I occasionally advise small teams on backend reliability, Go performance, and production AI systems. Learn more: /services