"Optimize for AMD Zen3+": The Microarchitecture Pivot That Unlocked a Performance Regression

In the middle of a deeply technical debugging session spanning Phases 0 through 4 of the cuzk project — a high-performance Groth16 proof generation pipeline for Filecoin PoRep — a single, seven-word user message cut through the noise: "Optimize for AMD Zen3+" ([msg 994]). This was not a command, not a question, and not a complaint. It was a strategic redirection. To understand why this message was written and why it matters, one must trace the thread of reasoning that led to it — a thread involving cache lines, stack frames, heap allocations, and a 10.5% performance regression that had stubbornly resisted explanation.

The Context: A Regression That Shouldn't Exist

The story begins with Phase 4 of the cuzk project, where five optimizations were applied to the proof generation pipeline in a single wave. The optimizations were labeled A1 (SmallVec for the LC Indexer), A2 (pre-sizing vectors), A4 (parallel B_G2 CPU MSMs), B1 (pinning a/b/c vectors with cudaHostRegister), and D4 (per-MSM window tuning). The baseline was a solid 88.9 seconds for a single 32 GiB PoRep proof. After applying all five changes, the total time ballooned to 106 seconds — a 19% regression.

Through disciplined instrumentation and systematic elimination, the assistant identified B1 as a major culprit: cudaHostRegister was pinning ~125 GiB of host memory, adding 5.7 seconds of overhead. Reverting B1 brought the total down to 94.4 seconds. But synthesis remained at 60.3 seconds, stubbornly 5.6 seconds above the 54.7-second baseline. The only remaining synthesis change was A1: replacing Vec<(usize, Scalar)> with SmallVec<[(usize, Scalar); 4]> in the Indexer struct of bellpepper-core's linear combination module.

This was baffling. SmallVec is designed to eliminate heap allocations for small collections by storing elements inline. For SHA-256-heavy PoRep circuits, where most linear combinations have 1–3 terms, SmallVec with an inline capacity of 4 should eliminate ~99% of heap allocations. Conventional wisdom says fewer allocations equals faster code. Yet the benchmark showed the opposite: a clear 10.5% slowdown.

The Assistant's Reasoning: Cache Lines and Stack Pressure

In [msg 991], the assistant began reasoning through the paradox. Each (usize, Scalar) tuple is 40 bytes (8 for the usize, 32 for the BLS12-381 Fr scalar). With INDEXER_INLINE_CAP = 4, each SmallVec stores 160 bytes inline. Each Indexer struct contains one SmallVec, and each LinearCombination contains two Indexers — so each LC is roughly 360+ bytes on the stack. During enforce(), three LCs (a, b, c) are created per constraint, totaling ~1080 bytes of stack data per call. In a parallel rayon synthesis loop running millions of constraints, this stack pressure could cause L1 data cache evictions.

The assistant considered whether the LCs actually had more than 4 terms (which would spill to heap and defeat the purpose), but concluded that for SHA-256 gadgets, 1–3 terms was typical. The real issue, the assistant hypothesized, was stack frame size: each Indexer grew from ~24 bytes (Vec: pointer + length + capacity) to ~170 bytes (SmallVec<4>: inline data + discriminant + length). Six Indexers per enforce() meant ~1020 bytes of inline data versus ~144 bytes with Vec — a 7x increase.

The assistant then posed a question to the user ([msg 991]), offering three paths: revert A1 entirely, try a smaller inline capacity of 2, or accept the regression. The user's response — "Fit to cache line? Consider CPU caches, maybe avx ops" — pointed toward microarchitecture-aware analysis.

The Subject Message: A Strategic Pivot

The assistant responded in [msg 992] with a detailed cache line analysis. A CPU cache line is 64 bytes. SmallVec<1> with 40 bytes inline fits in one cache line. SmallVec<2> at 80 bytes crosses a boundary. SmallVec<4> at 160 bytes spans three cache lines. Six Indexers at cap=4 spread ~960 bytes across ~15 cache lines — significant L1 pressure. The assistant decided to try cap=1 and edited the constant.

Then came the subject message ([msg 994]): "Optimize for AMD Zen3+".

This message was written because the user recognized that the assistant's analysis, while technically sound, was still reasoning in generic terms. The assistant had calculated cache line occupancy and stack frame sizes, but had not yet accounted for the specific characteristics of the target CPU — an AMD Zen4 Threadripper PRO 7995WX. The user was steering the investigation toward microarchitecture-specific optimization: not just "how many cache lines does this use," but "how does this specific CPU's L1d cache, prefetcher, and branch predictor interact with the SmallVec change?"

Assumptions and Knowledge Required

The subject message rests on several layers of prior knowledge. First, one must understand that the test system uses an AMD Zen4 architecture (Threadripper PRO 7995WX), which has specific cache geometries: 32 KB L1d per core (8-way, 64-byte lines), 1 MB L2 per core, and up to 32 MB L3 per 8-core CCD. Second, one must know that Zen3+ (and Zen4) has excellent branch prediction and hardware prefetchers that can make heap allocation patterns via jemalloc surprisingly fast — potentially faster than inline storage that blows out the L1d working set. Third, one must grasp the subtle trade-off: SmallVec eliminates pointer chases to heap data (which would miss L1 but likely hit L2), but at the cost of making the Indexer struct itself much larger, which increases cache footprint in the hot enforce() loop.

The message also assumes that the assistant has the tools and data to act on this directive — specifically, that the synth-only microbenchmark can be built and run quickly, and that hardware performance counters (perf stat) can be collected to measure L1/L2/L3 cache misses, branch mispredictions, and instructions per cycle.

The Thinking Process Revealed

The assistant's response in [msg 995] shows how the "Optimize for AMD Zen3+" directive was interpreted. The assistant immediately enumerated Zen3+ cache characteristics and connected them to the SmallVec analysis:

"With cap=4, each Indexer is ~170 bytes. 6 Indexers per enforce() = ~1020 bytes of hot stack data. That's 16 cache lines in L1d — not terrible but it thrashes the L1d working set when combined with the enforce() logic accessing constraint system vectors."

But then came a crucial insight — one that the user's directive likely intended to provoke:

"But actually, let me reconsider: Zen3+ has excellent branch prediction and prefetching. The Vec path with heap allocation goes through jemalloc's thread-local cache, which on Zen3+ is very fast (~10-15ns per alloc). The pointer chase to heap data would miss L1 but hit L2 (512KB) since the thread-local arena is hot. So Vec on Zen3+ may not be as slow as the proposal assumed."

This is the key realization that the user's message enabled. The original optimization proposal (from which A1 was drawn) assumed a generic cost model for heap allocation — roughly 15ns per allocation plus cache miss penalties. But on AMD Zen3+, jemalloc's thread-local caching makes small allocations extremely fast, and the L2 cache (1 MB per core on Zen4) is large enough to keep hot heap data resident. The SmallVec "optimization" was actually pessimizing for this specific microarchitecture by trading a cheap pointer chase for expensive stack cache pollution.

Output Knowledge Created

The subject message produced several concrete outcomes. First, it shifted the debugging strategy from generic performance analysis to microarchitecture-specific investigation. Second, it led to the decision to build a synth-only microbenchmark (in [msg 999]) that could isolate the SmallVec effect without GPU overhead, enabling rapid A/B testing. Third, it set the stage for collecting perf stat hardware counters to definitively measure whether cache misses were the root cause.

The message also implicitly validated the assistant's cache-line reasoning while pushing it further: yes, cache lines matter, but the specific CPU's cache hierarchy, prefetching behavior, and allocator performance determine whether inline storage is actually a win. On a different CPU — say, an Intel Xeon with smaller L2 caches or a different prefetcher — SmallVec might indeed be faster. The optimization must be evaluated on the target hardware.

Conclusion

"Optimize for AMD Zen3+" is a masterclass in concise technical direction. In seven words, the user reframed a puzzling regression as a microarchitecture-specific question, steering the investigation away from generic "SmallVec should be faster" assumptions and toward hardware-aware analysis. The message demonstrates that performance optimization is never purely algorithmic — it is always a dialogue between code and the silicon it runs on. The most sophisticated data structure change is meaningless if it doesn't account for the specific cache geometry, prefetcher behavior, and allocator performance of the target CPU. This single sentence transformed a debugging session into a lesson in microarchitecture-aware engineering.