The 12-GiB Opportunity: Early Deallocation of NTT Vectors in a GPU Proving Pipeline

In message [msg 3064] of a deep optimization session for the SUPRASEAL_C2 Groth16 proof generation pipeline, the assistant executed a single edit: freeing the massive a, b, and c NTT evaluation vectors immediately after the GPU's work completed, rather than holding them throughout the background b_g2_msm computation. The message itself is deceptively simple — a tool call result reading [edit] /home/theuser/curio/extern/bellperson/src/groth16/prover/supraseal.rs Edit applied successfully. — but the reasoning that led to this edit represents a masterclass in memory optimization at the intersection of Rust, C++, and CUDA.

The Question That Sparked the Investigation

The story begins with the user's incisive question in [msg 3055]: "the pending proof handle has nothing that can be freed early?" This came after the assistant had just committed Phase 12 of the optimization effort — a split GPU proving API that decoupled the b_g2_msm CPU computation from the GPU worker loop, achieving a 2.4% throughput improvement (37.1s/proof vs 38.0s baseline). However, Phase 12 had a hidden cost: the PendingProofHandle now held synthesis data approximately 1.7 seconds longer (during the background b_g2_msm), amplifying memory pressure. Attempts to increase synthesis parallelism (partition_workers=12) had resulted in out-of-memory (OOM) failures with RSS peaking at 668 GiB on a 755 GiB system.

The user's question was a challenge: was the handle holding memory unnecessarily? Could the memory footprint be reduced without sacrificing the throughput gains of the split API?

Tracing the Data Dependencies

The assistant's response in [msg 3056] began by reading the Rust source for PendingProofHandle, then systematically traced through the C++ CUDA code to determine exactly what the background prep_msm_thread actually needed. This was a critical investigative step — the assistant needed to understand the data flow across the Rust/C++ FFI boundary and the CUDA execution model.

The key insight came from examining the C++ function generate_groth16_proofs_start_c. The assistant used awk commands to grep through the CUDA source file (groth16_cuda.cu), searching for references to prover.a, prover.b, and prover.c within the prep_msm_thread lambda. The result was striking: these fields appeared only in lines 847–1195 of the CUDA file, which comprised the GPU kernel region — cudaHostRegister, cudaMemcpyAsync, cudaHostUnregister calls. All of these operations completed before the C++ function returned to Rust.

The background prep_msm_thread (which computes the b_g2_msm multi-scalar multiplication) needed only:

• inp_assignment_data and aux_assignment_data (the variable assignments)
• Density bitvectors (a_aux_density, b_inp_density, b_aux_density)
• Size metadata already copied into the heap-allocated pp->provers_owned struct It did not need the NTT evaluation vectors a, b, and c — each approximately 4 GiB (130 million 32-byte field elements), totaling ~12 GiB per partition.

The Memory Arithmetic

The assistant calculated the memory footprint precisely. A single partition's ProvingAssignment struct contains:

• a: Vec<Scalar> — ~4 GiB
• b: Vec<Scalar> — ~4 GiB
• c: Vec<Scalar> — ~4 GiB
• input_assignment: already moved out (empty)
• aux_assignment: already moved out (empty)
• Density trackers: ~16 MB each (negligible) The input_assignments and aux_assignments stored separately in the handle as Vec<Arc<Vec<Fr>>> were still needed — each aux_assignment was ~4 GiB, but the input_assignment was tiny (~10 KB). So the handle held approximately 16 GiB per partition total, of which ~12 GiB (the a/b/c vectors) could be freed immediately. The arithmetic was compelling: freeing 12 GiB per partition across the in-flight partitions (typically 10–15 with partition_workers=10 or 12) could reduce peak memory by 120–180 GiB — potentially enough to enable pw=12 without OOM.

The Edit Itself

Message [msg 3064] is the implementation of this insight. The edit to supraseal.rs added code in prove_start to clear the a, b, and c vectors from each prover immediately after the C++ start_groth16_proof function returned. The reasoning was sound: by the time generate_groth16_proofs_start_c returns, cudaHostUnregister has been called for all three vectors, and CUDA guarantees that unregistered host memory is no longer accessed by the device. The GPU kernels that read these vectors — the NTT evaluations and MSM computations — have completed. The background thread that continues running (prep_msm_thread) does not touch them.

The Rust code to free these vectors is straightforward: provers[c].a.clear(); provers[c].b.clear(); provers[c].c.clear(); — or more idiomatically, setting them to empty Vecs. But the impact is disproportionate to the code change. This is a pattern familiar to systems programmers: the most impactful optimizations often come not from complex algorithmic changes but from understanding the lifetime of data and releasing it at the earliest possible moment.

Safety Considerations

The assistant's investigation implicitly relied on several correctness assumptions. First, that the CUDA runtime's cudaHostUnregister truly ensures the device has finished accessing the memory. This is guaranteed by the CUDA API specification — cudaHostUnregister blocks until all pending operations on the registered memory complete. Second, that the C++ function's return implies all GPU kernel launches have finished. This is true because the code uses synchronous CUDA calls (or explicit synchronization points) before the unregister calls. Third, that the Rust compiler's optimizer would not reorder the clear operations before the C++ function call — this is guaranteed by Rust's evaluation order and the fact that the C++ call consumes references to the vectors.

The Broader Context

This edit was not an isolated optimization. It was part of a sustained, multi-phase effort documented across dozens of messages and several design proposals. The Phase 12 split API (committed at 99c31c2c) had already restructured the entire GPU proving workflow. The early deallocation was a refinement that addressed a memory regression introduced by that restructuring.

The assistant's todo list after the edit shows the next steps: rebuild and benchmark with early a/b/c free, try pw=12 again with reduced memory pressure, and commit the improvement. This demonstrates a disciplined engineering workflow — measure, identify, fix, verify.

What This Message Teaches

Message [msg 3064] exemplifies several principles of high-performance systems engineering. First, the importance of understanding data lifetimes across language boundaries — what the Rust side holds may not be what the C++ side needs, especially when background threads are involved. Second, the value of precise instrumentation: the assistant didn't guess about memory usage; it traced the exact code paths and calculated exact sizes. Third, the leverage of domain knowledge about the CUDA execution model — knowing that cudaHostUnregister provides a synchronization point allowed the assistant to safely free memory earlier than a naive reading of the code would suggest.

The edit itself is small — a handful of lines in a single file. But the reasoning that produced it spans hundreds of lines of code analysis, multiple source files in two languages, and a deep understanding of GPU memory management. It's a reminder that in systems programming, the most valuable optimizations often come not from writing new code but from understanding when existing data can be released.