The Art of Reverting: How a 5.7-Second Surprise Forced a Disciplined Retreat

The subject message is deceptively simple:

[assistant] [edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu Edit applied successfully.

On its surface, this is a mundane tool output — a confirmation that an edit was applied to a CUDA source file. But this message represents a pivotal moment in a disciplined performance engineering campaign. It is the quiet sound of a wrong turn being corrected, the digital equivalent of a surgeon removing a suture that was causing more harm than good. To understand why this single line matters, we must unpack the chain of reasoning, measurement, and analysis that led to it.

The Context: Phase 4 Optimization Wave

The assistant had been working on the cuzk project — a high-performance Groth16 proof generation pipeline for Filecoin's Proof-of-Replication (PoRep) protocol. Phases 0 through 3 had been successfully completed, establishing a strong baseline of 88.9 seconds for a single 32 GiB PoRep proof. Phase 4 was intended to be a wave of compute-level optimizations, implementing five changes drawn from a detailed optimization proposal document:

The Discovery: B1's Hidden Cost

The instrumented test run (see [msg 962]) produced the first clear phase-level breakdown of the GPU pipeline. The data was unambiguous:

| Phase | Time | |---|---| | pin_abc (B1) | 5,733 ms | | prep_msm | 1,722 ms | | ntt_msm_h | 21,417 ms | | batch_add | 1,446 ms | | tail_msm | 1,259 ms | | b_g2_msm | 22,849 ms | | Total proof | 101.3 s |

The cudaHostRegister pinning operation — intended to accelerate DMA transfers by registering ~125 GiB of host memory as pinned (page-locked) — was consuming 5.7 seconds all by itself. The optimization proposal had estimated this would cost 150–300 milliseconds. The actual cost was 19 times worse.

Why the enormous discrepancy? The cudaHostRegister call does not merely set a flag in the virtual memory manager. On Linux, it calls mlock() under the hood, which forces every page of the registered memory range to be faulted into RAM and locked there. When that memory range is 125 GiB — spanning 10 circuits × 3 arrays (a, b, c) × ~4.17 GiB each — the kernel must touch and lock approximately 32 million 4 KiB pages. This page-fault storm dominates the 5.7-second cost, far outweighing any potential DMA bandwidth benefit.

The Reasoning: Why Revert?

The assistant's analysis in [msg 968] laid out the verdict with clarity:

B1 (cudaHostRegister) is the BIGGEST problem: 5.7 seconds just for pinning! That's ~125 GB being mlock'd. This alone accounts for most of the GPU phase regression (40.1s vs 34.0s baseline). The proposal estimated 150-300ms but it's actually 19x worse because mlock on 125 GB touches every page.

The decision to revert was not impulsive. It was based on a careful cost-benefit calculation:

  1. Cost: 5.7 seconds of pinning overhead, plus additional unpinning overhead at the end
  2. Benefit: Potentially faster cudaMemcpyAsync transfers, estimated to save at most 0.3–0.5 seconds
  3. Net: A clear loss of ~5 seconds The assistant also noted that the GPU compute itself was actually performing well — the pure GPU path (NTT + batch addition + tail MSMs) completed in 24.1 seconds, which was 10 seconds faster than the baseline's 34 seconds. This suggested that D4 (per-MSM window tuning) and other changes were genuinely helping the GPU computation. The problem was exclusively the pinning overhead.

The Edit: Completing the Reversion

The subject message ([msg 973]) is the second of two edits that together revert B1. The first edit ([msg 970]) removed the pinning code block at the beginning of the generate_groth16_proofs_c function. This second edit removes the unpinning code at the end:

// B1: Unpin a,b,c vectors
for (size_t c = 0; c < num_circuits; c++) {
    cudaHostUnregister((void*)provers[c].a);
    cudaHostUnregister((void*)provers[c].b);
    cudaHostUnregister((void*)provers[c].c);
}

Both the pin and unpin operations had to be removed as a pair. Leaving the unpin code without the corresponding pin would have caused undefined behavior — calling cudaHostUnregister on memory that was never registered. The assistant's approach was methodical: first remove the pin block, then search for and remove the unpin block, ensuring the codebase remained consistent.## Assumptions Under the Microscope

This episode reveals several assumptions that proved incorrect:

Assumption 1: cudaHostRegister overhead is proportional to the number of allocations, not the total memory size. The optimization proposal estimated 150–300 ms based on the assumption that pinning a handful of large buffers would be cheap. In reality, cudaHostRegister on Linux calls mlock() which must walk and fault every page in the range. For 125 GiB, that means touching ~32 million pages — an operation that scales with the memory size, not the number of allocations.

Assumption 2: The DMA bandwidth improvement would outweigh the pinning cost. The reasoning was that without pinned memory, cudaMemcpyAsync falls back to an internal 32 MiB staging buffer, cutting effective bandwidth by roughly half. For ~40 GiB of GPU transfers, the expected saving was 0.3–0.5 seconds. But the 5.7-second pinning cost dwarfed any possible transfer benefit.

Assumption 3: The optimizations could be evaluated as a set. The initial 106-second result lumped all five changes together, making it impossible to identify individual contributions. This motivated the instrumentation effort — without fine-grained timing data, the B1 regression would have remained hidden inside the aggregate slowdown.

Assumption 4: SmallVec would be a pure win. The A1 optimization (replacing Vec with SmallVec for the LC indexer) was expected to reduce heap allocations and improve synthesis time. Yet the synthesis phase remained 5–6 seconds slower even after reverting A2 and B1, pointing to A1 as the next suspect. This assumption would be tested in subsequent microbenchmarks ([msg 1008]), which confirmed that SmallVec caused a consistent ~5–6 second regression across all inline capacities (1, 2, 4 elements) compared to the original Vec.

Input Knowledge Required

To understand this message and the reasoning behind it, one needs:

Output Knowledge Created

This message and the analysis that preceded it produced several lasting insights:

  1. B1 is net-negative for this workload: cudaHostRegister on ~125 GiB costs 5.7 seconds, making it worse than the unoptimized baseline. The optimization should not be used in the current pipeline architecture.
  2. GPU compute is genuinely faster: The pure GPU path (NTT + batch addition + tail MSMs) improved from 34 seconds to 24.1 seconds, likely due to D4 per-MSM window tuning. This is a real win worth keeping.
  3. The synthesis regression is separate: With A2 and B1 reverted, the remaining 5.5-second gap above the 88.9-second baseline pointed squarely at A1 (SmallVec). This isolated the problem for targeted investigation.
  4. Instrumentation is essential: Without the CUZK_TIMING printf's, the B1 cost would have remained invisible inside the aggregate GPU time. The investment in adding fine-grained timing to the CUDA code paid for itself in a single test run.
  5. A disciplined revert strategy: The assistant preserved the other optimizations (A1, A4, D4) while reverting only the proven harmful change. This incremental approach — revert one thing, measure, decide — is a textbook example of scientific performance engineering.

The Broader Narrative

The B1 reversion is not an admission of failure. It is a demonstration of intellectual honesty and methodological rigor. In performance engineering, the most important skill is not knowing which optimizations to apply — it is knowing which optimizations to remove. Every optimization carries a cost: code complexity, maintenance burden, and sometimes hidden runtime overhead. The willingness to measure, analyze, and revert is what separates disciplined engineering from cargo-cult optimization.

This single edit message — "[edit] groth16_cuda.cu / Edit applied successfully" — represents the culmination of a diagnostic chain that began with a 106-second regression, passed through instrumentation, buffer-flushing fixes, build-system debugging, and careful timing analysis. It is the quiet sound of a system being made better by having something taken away.

The story does not end here. After reverting B1, the total proof time dropped from 101.3 seconds to 94.4 seconds — an improvement, but still 5.5 seconds above the 88.9-second baseline. The synthesis regression remained, and the assistant would go on to build a synth-only microbenchmark to isolate the A1 (SmallVec) problem, ultimately discovering that SmallVec caused a consistent 5–6 second slowdown on the AMD Zen4 Threadripper PRO 7995WX system. But that is the next chapter. For now, the lesson is clear: sometimes the best optimization is the one you choose not to keep.