Diagnosing a Performance Regression: The SmallVec Mystery on AMD Zen4

Deep in the optimization trenches of a Groth16 proof generation pipeline for Filecoin's Proof-of-Replication (PoRep), the assistant faces a puzzle. An optimization that should have improved performance has instead made things worse. The culprit is the A1 optimization from Phase 4 of the project: replacing Vec with SmallVec in the LinearCombination indexer, hoping to improve cache behavior by storing small vectors inline rather than on the heap. Microbenchmarks have already shown that SmallVec with an inline capacity of 2 is slower than plain Vec on the AMD Zen4 target machine — a counterintuitive result that demands explanation. Message [msg 1065] captures a pivotal moment in this investigation: the second of two perf stat runs collecting hardware performance counter data for the SmallVec configuration, before reverting to Vec to collect a comparison baseline.

The Optimization Journey So Far

The broader project spans four phases of optimization for the cuzk proving engine. Phase 1 established a pipelined synthesis/GPU prover for PoRep C2 proofs. Phase 2 extended this to all proof types with batch-mode synthesis and async overlap between CPU synthesis and GPU proving, achieving a 1.27× throughput improvement. Phase 3 introduced cross-sector batching, pushing throughput to 1.46×. Phase 4, the current phase, targets compute-level micro-optimizations: four CPU-side changes (A1–A4), one GPU memory optimization (B1), and one MSM window-tuning change (D4).

The A1 optimization — replacing Vec with SmallVec in the LC Indexer — was motivated by a well-known principle: small vectors that fit in a cache line can be stored inline, avoiding heap allocation overhead and improving cache locality. On Intel architectures, this pattern often yields significant wins. But on the AMD Zen4 machine used for development and benchmarking, the opposite occurred. Every configuration of SmallVec (inline capacities of 1, 2, and 4) was slower than the original Vec implementation. This contradicted both intuition and the Intel-biased conventional wisdom, forcing a deeper investigation.

The Message: Collecting Hardware Counter Data

The subject message is concise but dense with intent. It begins with the assistant's assessment: "Excellent data! SmallVec cap=2 baseline captured. Now let me do a second run for consistency, then revert to Vec and rebuild." This single sentence reveals the methodological approach — the assistant is not satisfied with a single data point and explicitly seeks consistency before making a potentially costly decision (reverting code and rebuilding the entire binary).

The command issued is a perf stat invocation targeting a carefully selected set of hardware performance counters:

perf stat -e instructions,cycles,cache-references,cache-misses,
  branch-instructions,branch-misses,
  l2_cache_req_stat.dc_access_in_l2,
  l2_cache_req_stat.dc_hit_in_l2,
  ls_dmnd_fills_from_sys.local_l2,
  ls_dmnd_fills_from_sys.local_ccx,
  ls_any_fills_from_sys.dram_io_all

Each counter was chosen to test a specific hypothesis about why SmallVec regresses on Zen4:

Scientific Rigor and the Consistency Check

The assistant's decision to run a second measurement "for consistency" reflects sound experimental practice. The first run ([msg 1064]) completed in 55.84 seconds. This second run completes in 57.10 seconds — a difference of roughly 1.3 seconds, or about 2.3%. This variance is notable and could arise from several sources: thermal throttling, background system activity, NUMA effects, or inherent variability in the perf stat multiplexing when measuring many hardware counters simultaneously.

The fact that the assistant flags this as a consistency check — rather than simply accepting the first result and moving on — demonstrates a mature understanding of performance measurement. In microbenchmarking, single-run results can be misleading. The variance between these two runs (55.84s vs 57.10s) itself tells a story: the measurement is noisy enough that a 2% difference between runs is normal, meaning any optimization claim below that threshold should be treated with skepticism.

What the Data Will Reveal

The perf stat output is captured to a file (/tmp/perf-smallvec-cap2-run2.txt) for later analysis alongside the Vec baseline. While the message does not include the actual counter values (those would appear in the tool output that follows in the next message), the choice of counters reveals the assistant's mental model of the regression.

The central hypothesis is that SmallVec's inline storage, while reducing heap allocations, introduces other costs on Zen4: perhaps the inline storage increases the size of the LinearCombination struct, causing more cache line fills per access; perhaps the branching logic for inline-vs-heap storage confuses the branch predictor; perhaps the smaller inline capacity (2 elements) forces frequent spillover to heap allocation anyway, adding complexity without benefit. The L2 and DRAM fill counters will show whether memory access patterns have changed, while the branch miss counter will reveal control-flow impacts.

Broader Implications

This message represents a critical inflection point in the optimization effort. The assistant is methodically testing assumptions and letting data drive decisions. The SmallVec regression on Zen4 is a cautionary tale about architecture-specific optimization: a technique that works well on Intel's big-little hybrid architectures or on server-class Xeons may not translate to AMD's unified CCX topology. The inline capacity of 2, which seemed like a safe bet for the LC Indexer (where most vectors are small), turns out to be a net loss.

The systematic approach on display here — formulate hypothesis, implement change, benchmark, collect hardware counters when results are unexpected, verify consistency, compare against baseline — is a model for performance engineering. The assistant does not blindly revert the change; instead, it seeks to understand why the change regressed, building knowledge that will inform future optimization decisions. Whether the data ultimately confirms the cache-pressure hypothesis or reveals an unexpected bottleneck, the knowledge gained from this investigation is valuable far beyond the immediate decision to keep or revert A1.

Conclusion

Message [msg 1065] captures a moment of disciplined performance analysis: the second of two consistency runs collecting hardware counter data for a puzzling regression. The assistant's methodical approach — using perf stat with targeted counters, running multiple iterations, and planning a controlled comparison against the Vec baseline — exemplifies how to diagnose subtle performance issues in systems where intuition and conventional wisdom can be misleading. The data collected here will inform one of the key decisions in Phase 4: whether to keep the A1 SmallVec optimization or revert it in favor of the simpler Vec implementation. More importantly, the understanding gained from this analysis will shape future optimization decisions across the entire proving pipeline.