The 5.7-Second Mistake: Reverting a cudaHostRegister Optimization That Cost 19x More Than Expected

In the high-stakes world of GPU-accelerated zero-knowledge proof generation, every millisecond counts. When the cuzk team set out to optimize their Groth16 proving pipeline for Filecoin's Proof-of-Replication (PoRep) protocol, they implemented five carefully designed optimizations intended to shave seconds off an already-optimized 88.9-second baseline. But when the dust settled after the first integrated test, the total time had increased to 106 seconds — a 17-second regression. Message [msg 970] captures the moment when the team executed the first decisive corrective action: reverting optimization B1, the cudaHostRegister memory pinning change, which was discovered to be adding 5.7 seconds of overhead instead of the estimated 150–300 milliseconds.

The Context: A Performance Regression Demands Surgical Diagnosis

The cuzk project is building a pipelined, memory-efficient SNARK proving engine for Filecoin storage proofs. By Phase 4, the team had already achieved a strong baseline of 88.9 seconds for a single 32 GiB PoRep proof through careful pipelining and cross-sector batching. Phase 4 aimed to push further with compute-level micro-optimizations, implementing five changes drawn from a detailed optimization proposal:

CUZK_TIMING: pin_abc_ms=5733 num_circuits=10 abc_bytes_each=4168923808
CUZK_TIMING: prep_msm_ms=1722
CUZK_TIMING: gpu_tid=0 ntt_msm_h_ms=21417
CUZK_TIMING: gpu_tid=0 batch_add_ms=1446
CUZK_TIMING: gpu_tid=0 tail_msm_ms=1259 gpu_total_ms=24123
CUZK_TIMING: b_g2_msm_ms=22849 num_circuits=10

The total proof time was 101.3 seconds — still 12.4 seconds above the baseline. But now the team had a phase-level breakdown that pinpointed the problem.

The Smoking Gun: 5.7 Seconds of Pinning Overhead

The pin_abc_ms=5733 line was the revelation. Optimization B1 called cudaHostRegister on the a, b, and c coefficient vectors for each of the 10 circuits in a PoRep proof. Each circuit's three vectors totaled approximately 4.17 GB, meaning the function was pinning roughly 125 GB of host memory. The cudaHostRegister call works by locking the virtual memory pages into physical RAM and updating the IOMMU mappings — an operation that touches every single page. At 125 GB, even at modern memory bandwidth speeds, this was an enormous amount of work.

The original optimization proposal had estimated the pinning overhead at 150–300 milliseconds, based on the assumption that cudaHostRegister was primarily a metadata operation. In reality, on the AMD Zen4 Threadripper PRO 7995WX system with its 8-channel DDR5 memory, the kernel-level page locking and TLB invalidation required to pin 125 GB of pages took 5.7 seconds — 19 times the upper estimate. The DMA bandwidth improvement from pinning, estimated at roughly 0.3–0.5 seconds of transfer time savings, was completely swamped by this upfront cost.## The Edit: A Single Line of Confirmation

The subject message itself — <msg id=970> — is deceptively simple:

[assistant] [edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu
Edit applied successfully.

This single line is the culmination of an intensive diagnostic chain spanning dozens of messages. It represents the moment when data-driven engineering discipline overruled theoretical optimization promise. The edit removed the cudaHostRegister calls that pinned the a, b, and c vectors before GPU transfer, and the corresponding cudaHostUnregister calls at the function's end (removed in the subsequent [msg 971] and [msg 972]). The decision was not made lightly — it was backed by hard data showing a 5.7-second penalty.

The assistant's reasoning, articulated in the analysis message [msg 968] that immediately preceded the edit, was methodical:

  1. B1 is the biggest problem: 5.7 seconds of pinning overhead for at most 0.3–0.5 seconds of DMA savings. Net negative.
  2. B_G2 MSM is slow but not on the critical path: At 22.8 seconds, it runs concurrently with GPU work and finishes before the GPU's 24.1-second internal compute. The A4 parallelization helped here.
  3. GPU compute is actually faster than baseline: The pure GPU path (NTT + batch + tail) at 24.1 seconds was 10 seconds faster than the 34-second baseline, suggesting D4's window tuning was effective.
  4. Synthesis still shows a regression: At 61 seconds versus the 54.7-second baseline, the A1 SmallVec change might be the culprit — but that would be investigated next. The verdict was clear: revert B1, keep A1, A4, D4, and re-test.## Assumptions, Mistakes, and Lessons Learned The B1 episode reveals several important assumptions that turned out to be incorrect. The first was the estimated cost of cudaHostRegister. The optimization proposal assumed 150–300 ms, likely extrapolating from small-scale benchmarks where pinning a few hundred megabytes is indeed cheap. But at the scale of 125 GB, the operation becomes a system-level event: the kernel must walk the page tables for every page, mark them as non-swappable, and update the IOMMU mappings. On a system with 125 GB of RAM, this is a memory-bandwidth-limited operation that takes seconds, not milliseconds. The second assumption was that the DMA bandwidth improvement from pinning would be significant enough to justify the cost. In the existing pipeline, the a, b, and c vectors were already being transferred via cudaMemcpyAsync with unpinned host memory, which CUDA handles by staging through an internal 32 MiB pinned buffer. While this staging does reduce throughput, the actual transfer time for the vectors was a small fraction of the total GPU time. The 0.3–0.5 seconds saved was dwarfed by the 5.7-second pinning tax. The third assumption was that the pinning could be done once and amortized across multiple proof jobs. In the current architecture, the cudaHostRegister calls happen inside generate_groth16_proofs_c, which is called fresh for each proof. The vectors are allocated by the Rust-side bellperson library and are not reused across calls, so the pinning cost is paid every time. A persistent memory pool with pre-pinned buffers could theoretically amortize this cost, but that would require a deeper architectural change than the Phase 4 scope allowed.

The Broader Engineering Philosophy

What makes this message noteworthy is not the edit itself — it's a one-line confirmation of a file change — but what it represents. The team had invested significant effort in implementing five optimizations, building instrumented CUDA code, debugging printf buffering, forcing CUDA recompilation, and running end-to-end GPU tests. The temptation to keep B1, to rationalize the 5.7 seconds as "acceptable overhead" or to blame the regression on other changes, must have been real. Instead, the data was allowed to speak: B1 was a net negative, and it had to go.

This discipline paid off. After reverting B1, the next test run (documented in subsequent messages) brought the total proof time down to 94.4 seconds — a 7-second improvement from the 101.3-second instrumented run, but still 5.5 seconds above the 88.9-second baseline. The remaining regression was traced to the A1 SmallVec change, which was then isolated using a synth-only microbenchmark that avoided the overhead of GPU proving and SRS loading entirely. By the end of the diagnostic chain, the team had identified that SmallVec, regardless of inline capacity, caused a consistent 5–6 second slowdown in synthesis time — a finding that contradicted the intuitive expectation that reducing heap allocations would speed things up.

Conclusion

Message [msg 970] is a masterclass in performance engineering discipline. It shows that the most important optimization skill is not knowing which changes to make, but knowing which changes to undo. The 5.7-second pinning overhead was invisible without precise instrumentation, and the decision to revert required trusting the data over the optimization proposal's estimates. In a field where every millisecond is fought for, the willingness to discard a well-intentioned optimization that doesn't deliver is what separates effective engineering from wishful thinking. The B1 reversion cleared the way for the real diagnosis — the A1 SmallVec regression — and ultimately led to a cleaner, more honest understanding of where the pipeline's bottlenecks truly lay.