The Dangling Reference: A Microscope on a Use-After-Free Bug in GPU Proving

In the high-stakes world of GPU-accelerated zero-knowledge proof generation, a single dangling pointer can corrupt hundreds of gigabytes of in-flight computation. This article examines a single message from an opencode coding session — message index 3012 — in which an AI assistant reads a Rust source file to verify the data flow behind a suspected use-after-free bug in the Phase 12 split GPU proving API for Filecoin's PoRep (Proof-of-Replication) pipeline. Though the message itself is brief — a file read showing seven lines of Rust code — it represents a pivotal moment in a debugging investigation that would ultimately uncover a critical concurrency bug and reshape the memory architecture of the proving system.

The Scene: A Memory Crisis

The conversation leading up to message 3012 is tense. The assistant has been implementing Phase 12 of a multi-phase optimization campaign for the cuzk SNARK proving engine — a system that generates Groth16 proofs for Filecoin storage proofs. Phase 12 introduced a "split API" that decouples the GPU worker's critical path from CPU post-processing: instead of blocking on the full GPU proof pipeline, the worker can start a proof (start_groth16_proof), hand off the pending work to a background thread, and immediately pick up the next partition. This should improve throughput by hiding the latency of the b_g2_msm (a multi-scalar multiplication on the G2 curve) behind other work.

But benchmarking reveals a problem. With the previous configuration (pw=10, meaning 10 concurrent partition syntheses), the system runs fine at ~37 seconds per proof. Bumping to pw=12 or pw=15 causes out-of-memory (OOM) errors, with RSS peaking at 668 GiB on a 755 GiB machine. The user flags this: "It's not 400GB tho, something is leaking somewhere" ([msg 2993]). The assistant initially suspects a memory leak — perhaps the PendingProofHandle is not being freed, or deallocation threads are stacking up.

Tracing the Data Flow

The investigation narrows to a critical question: does the C++ background thread (prep_msm_thread, which performs the b_g2_msm computation) still reference Rust-owned memory after start_groth16_proof returns? If so, the Rust data must stay alive until the thread completes. But if the Rust data is freed too early — or if the C++ code holds a dangling reference — the system could crash or corrupt data.

In message 3011, the assistant reads the C++ CUDA source (groth16_cuda.cu) and discovers something alarming. The prep_msm_thread is spawned inside generate_groth16_proofs_start_c with a capture clause [&, num_circuits] — capturing all local variables by reference. One of those locals is provers, a std::vector<Assignment> constructed on the stack from the Rust-provided pointers. The assistant writes:

"But wait — provers is a local variable on the stack in generate_groth16_proofs_start_c. It's a vector<Assignment> built from the Rust-passed pointers. The prep_msm_thread captures &provers by reference. When generate_groth16_proofs_start_c returns, the provers local goes out of scope — the reference is dangling!"

This is the smoking gun. But the assistant needs to confirm the full chain — specifically, how the Rust side constructs and passes the provers array to the C++ FFI. That is exactly what message 3012 does.

The Subject Message: Reading the Rust Side

Message 3012 is the assistant issuing a read tool call on the file /home/theuser/curio/extern/supraseal-c2/src/lib.rs, showing lines 270 through 277:

270:         "only 64-bit elements in bit vectors are supported"
271:     );
272:     assert!(a_aux_density_bv.len() * bv_element_size >= aux_assignments_size);
273:     assert!(b_g1_aux_density_bv.len() * bv_element_size >= aux_assignments_size);
274: 
275:     let provers: Vec<_> = (0..num_circuits)
276:         .map(|c| Assignment::<S> {
277:             a_aux_density: a_aux_density_bv.as_ptr() as *cons...

The content is truncated (the line cuts off at *cons...), but the critical structure is visible. On line 275, provers is declared as a local Vec — a Rust Vec<Assignment<S>> constructed by mapping over circuit indices. Each Assignment is populated with pointers into the bit-vector density data (a_aux_density_bv.as_ptr()) and, presumably, the input and auxiliary assignment data.

This confirms the data flow: Rust builds a Vec<Assignment> on the stack, passes its raw pointer to C++ via FFI, and C++ receives it as a const Assignment<fr_t> provers[] parameter. The C++ function then copies this into its own local vector<Assignment> provers — but the background thread captures a reference to that local vector, not a copy. When the C++ function returns, the local vector is destroyed, and the background thread is left with a dangling reference to freed memory.

Input Knowledge Required

To understand the significance of this message, one must grasp several layers of the system architecture. First, the FFI boundary between Rust and C++: Rust owns the original assignment data (the circuit values, density bit-vectors, and scalar field elements), while C++ receives raw pointers to this data. Second, the ownership semantics of std::thread in C++: a thread spawned with [&] capture captures references to the spawning function's locals, and those references become dangling once the function returns. Third, the Groth16 proof pipeline itself: the b_g2_msm computation reads the prover's assignment data (specifically inp_assignment_data and aux_assignment_data) to construct the MSM inputs, so the data must remain valid for the thread's duration.

The assistant also needs to understand the memory layout: each partition's synthesis produces ~13 GiB of assignment data (provers, input assignments, auxiliary assignments). With pw=10, ten partitions are in flight simultaneously, plus two GPU workers each holding a pending proof handle with its ~13 GiB of data. The 755 GiB system can just barely accommodate this. Adding two more concurrent syntheses (pw=12) pushes past the limit — but the root cause isn't just the raw memory; it's that the dangling reference bug means the system is holding data longer than necessary, because the Rust side cannot safely free it until the background thread completes.

Output Knowledge Created

This single read operation produces a crucial piece of evidence: confirmation that provers is a Rust local variable, constructed inline and passed by pointer to C++. This completes the picture of the use-after-free bug. The assistant now knows:

1. Rust side: provers is a Vec<Assignment> local in the Rust FFI function (likely start_groth16_proof in lib.rs). It is constructed from the density bit-vectors and passed to C++ via as_ptr().
2. C++ side: The C++ function generate_groth16_proofs_start_c receives this as a pointer, builds its own local vector<Assignment>, spawns a background thread that captures &provers (reference to the C++ local), and then returns — destroying the local vector while the thread still runs.
3. The fix: The provers array must be copied into the heap-allocated groth16_pending_proof struct so that the background thread accesses stable memory. This is exactly what the assistant implements in the subsequent messages: adding provers_owned to the pending proof struct and copying the provers array into it before spawning the thread.

The Thinking Process Visible

What makes this message fascinating is not what it says but what it reveals about the assistant's investigative methodology. The assistant is systematically tracing the data flow across the Rust/C++ boundary, following pointers from one side to the other. In message 3011, it reads the C++ code and spots the dangling reference. In message 3012, it immediately turns to the Rust side to confirm the provenance of the provers array. This is classic debugging: form a hypothesis (the background thread has a dangling reference), then gather evidence from both sides of the interface to confirm it.

The assistant also demonstrates a deep understanding of undefined behavior. A dangling reference in C++ doesn't always cause an immediate crash — it depends on whether the memory has been reused and whether the thread reads from it before the system reallocates the page. The assistant's concern is not just about crashes but about correctness: if the background thread reads stale or corrupted data, the proof will be invalid, and the system will produce incorrect results silently.

Assumptions and Potential Mistakes

The assistant makes one key assumption that deserves scrutiny: that the dangling reference is the primary cause of the OOM at pw=12. In reality, the OOM has two contributing factors. First, the dangling reference means the Rust side cannot safely free the assignment data until the background thread completes — but the Rust side doesn't know when the thread completes because the split API has separated start from finish. The PendingProofHandle holds the data alive until finish_pending_proof is called, which happens only after the background thread joins. So the data is held for the correct duration — the bug is that the C++ side accesses it through a dangling reference, not that the data is freed too early.

Second, the OOM at pw=12 is primarily a capacity issue: the system simply has too many partitions in flight simultaneously, each holding ~13 GiB of data. The dangling reference bug is a correctness issue (potential use-after-free) but not necessarily the memory culprit. The assistant later discovers this distinction and addresses both problems separately: fixing the dangling reference by copying provers into heap memory, and addressing the memory pressure by tuning the channel capacity between synthesis and GPU workers.

The Broader Significance

Message 3012 is a small but essential step in a larger debugging narrative. It represents the moment when the assistant moves from suspicion to confirmation — from "this might be a dangling reference" to "here is exactly how the data flows, and here is where the bug lives." This kind of cross-language debugging is notoriously difficult because the Rust and C++ memory models differ, and the FFI boundary obscures ownership semantics. The Rust side assumes that passing a pointer to C++ transfers responsibility for the data's lifetime to the C++ side. The C++ side assumes that a reference capture in a lambda keeps the data alive. Neither assumption is wrong in isolation, but together they create a contradiction that neither language's safety guarantees can catch.

The fix — copying the provers array into the heap-allocated pending proof struct — is a textbook solution to this class of problem. It makes the background thread's data lifetime independent of the spawning function's stack frame, ensuring that the thread always accesses valid memory regardless of when the function returns. This pattern (heap-allocate shared state, never capture stack references in asynchronous lambdas) is a fundamental rule of safe concurrent programming in C++, one that is easy to violate when refactoring synchronous code into asynchronous code — exactly what Phase 12 did.

Conclusion

Message 3012 is a quiet but critical moment in a complex debugging session. A single file read operation, showing seven lines of Rust code, provides the missing piece of evidence needed to confirm a use-after-free bug in a high-performance GPU proving system. The message exemplifies the meticulous, evidence-driven approach required to debug cross-language concurrency issues: form a hypothesis, trace the data flow on both sides of the FFI boundary, verify the ownership semantics, and then design a fix that makes the lifetimes explicit and safe. In the end, the dangling reference was fixed, the memory pressure was addressed through channel capacity tuning, and Phase 12 delivered a working split API with a 2.4% throughput improvement — but only after this crucial moment of understanding.