The Moment the Provers Array Dangled: A Use-After-Free Bug Discovered in a Split GPU Proving API

In the high-stakes world of GPU-accelerated cryptographic proof generation, memory safety is not a luxury — it is a necessity. When a background thread reads from a stack-allocated array whose owning function has already returned, the result is undefined behavior: corrupted data, silent data loss, or a crash that surfaces only under specific timing conditions. This is precisely the kind of bug that the assistant discovered in message [msg 3015] of an opencode coding session, while implementing Phase 12 of a split GPU proving API for the SUPRASEAL_C2 Groth16 proof generation pipeline used in Filecoin's Proof-of-Replication (PoRep).

The message is brief — barely a few lines of reasoning and a read command — but it represents a critical turning point. It is the moment when a vague suspicion crystallizes into a concrete, confirmed bug. The assistant had been investigating memory pressure issues, watching RSS climb to 367 GiB during benchmarks, and wondering whether the PendingProofHandle was holding onto synthesis data longer than necessary. But in this message, the assistant realizes something far more alarming: the provers array, a C++ parameter pointing into Rust-owned memory, is being accessed by a background thread after the function that owns it has returned. This is a use-after-free bug, and it has been silently lurking in the code.

The Context: Phase 12 and the Split API

To understand why this bug matters, we must first understand what Phase 12 was trying to accomplish. The SUPRASEAL_C2 pipeline generates Groth16 proofs for Filecoin sectors — a computationally intensive process that involves multiple partitions, each requiring synthesis (circuit evaluation) followed by GPU-accelerated multiscalar multiplications (MSMs) and number-theoretic transforms (NTTs). The critical path through the GPU worker included a step called b_g2_msm — a G2-group MSM that consumed approximately 1.7 seconds of GPU time. In the original synchronous API, the GPU worker would block for this entire duration, holding all partition resources and preventing the next partition from starting.

Phase 12 introduced a split API: start_groth16_proof would launch the GPU kernel region and spawn a background thread (prep_msm_thread) to run b_g2_msm asynchronously, returning immediately with a PendingProofHandle. A separate finish_pending_proof function would later join the background thread and assemble the final proof. This decoupling allowed the GPU worker to begin processing the next partition sooner, improving throughput by approximately 2.4% (from 38.0s to 37.1s per proof).

But this architectural change introduced a subtle lifetime problem. In the old synchronous path, generate_groth16_proofs_c called generate_groth16_proofs_start_c (which spawned the prep_msm_thread) and then immediately called finalize_groth16_proof_c (which joined the thread). The provers array — a C-style array of Assignment<fr_t> structs passed as a function parameter — remained valid throughout because the function did not return until the thread completed. In Phase 12, generate_groth16_proofs_start_c returns while the background thread is still running. The provers parameter, being a stack-local pointer in the C++ function's frame, becomes a dangling reference the moment the function returns.

The Discovery: Tracing the Provers Array

The assistant's journey to this discovery began in the preceding messages ([msg 2996] through [msg 3014]), where the focus was on understanding why memory was accumulating during benchmark runs. RSS was peaking at 367 GiB with 15 concurrent jobs and 10 partitions each, and the assistant was trying to determine whether the PendingProofHandle was holding onto synthesis data (the massive a, b, c NTT evaluation vectors, each ~12 GiB per partition) longer than necessary.

The assistant traced the C++ prep_msm_thread and confirmed that it reads from Rust-owned memory: provers[c].inp_assignment_data, provers[c].aux_assignment_data, and density bit-vectors. These are raw pointers into Rust Vec<Arc<Vec<Fr>>> allocations. The assistant initially thought the issue was merely about memory accumulation — that the split API was keeping synthesis data alive for an extra ~1.7 seconds while the background thread ran, causing more partitions to be in-flight simultaneously.

But then came the crucial insight in [msg 3011]: "provers is a local variable on the stack in generate_groth16_proofs_start_c. It's a vector<Assignment> built from the Rust-passed pointers. The prep_msm_thread captures &provers by reference. When generate_groth16_proofs_start_c returns, the provers local goes out of scope — the reference is dangling!"

This was the first articulation of the bug. The assistant immediately followed up by examining the Rust side ([msg 3013]), confirming that the Rust start_groth16_proof function builds a local provers: Vec<Assignment<S>> and passes its buffer pointer to C++. When start_groth16_proof returns, the Rust Vec is dropped, and the memory is freed. The C++ background thread continues reading from that freed memory.

The Subject Message: Confirmation and Deepening Concern

Message [msg 3015] is the moment of confirmation. The assistant writes:

Wow, provers is used extensively — not just in prep_msm_thread but also in the main body (GPU kernel region, lines 825-1177). The prep_msm_thread runs in parallel with the GPU kernel region. So provers must stay alive through the entire function execution.

This is a critical realization. The provers array is not merely accessed by the background thread — it is also read by the main GPU kernel region at lines 825-1177 of groth16_cuda.cu. The code at line 825 reads provers[0].abc_size, and line 829 calls cudaHostRegister on provers[0].a, provers[0].b, and provers[0].c. These are the NTT evaluation vectors — the massive ~12 GiB buffers that hold the circuit's A, B, and C polynomials. The main body also calls cudaHostUnregister on these pointers at lines 1175-1177, just before the function returns.

The assistant then reads the function's tail to find exactly where it returns and what state the provers array is in at that point. The read command targets lines 1160-1165, showing the cleanup code that frees CUDA resources (device buffers, events, streams) while still holding the GPU lock.

This message is notable for what it does not contain: a fix. The assistant does not yet propose a solution. Instead, the message is purely diagnostic — a deepening of understanding. The assistant is mapping the terrain, confirming that the bug is real and understanding its full scope. The fix will come in subsequent messages: copying the provers array into the heap-allocated groth16_pending_proof struct as a provers_owned field, ensuring the background thread always accesses stable memory.

Technical Depth: Why This Is a Use-After-Free

The bug is subtle and worth examining in detail. In C++, a function parameter declared as const Assignment<fr_t> provers[] decays to const Assignment<fr_t>* provers — a pointer to the first element of the array. The lambda [&, num_circuits] captures all local variables by reference, including provers. But provers is a pointer variable on the stack. Capturing it by reference means the lambda holds a reference to the stack slot that holds the pointer value — not a copy of the pointer value itself.

When generate_groth16_proofs_start_c returns, its stack frame is popped. The stack memory that held the provers pointer variable is now available for reuse. The background thread, still running, dereferences its reference to this stack slot, reads whatever value is now there (potentially garbage), and then dereferences that as a pointer to Assignment<fr_t> structs. Even if the reference to the stack slot still happens to hold the original pointer value (because the memory hasn't been overwritten yet), the underlying data that the pointer points to — the Rust Vec buffer — has already been freed when start_groth16_proof returned.

This is a classic use-after-free, and it is particularly dangerous because it may appear to work correctly under most conditions. The stack memory may not be immediately overwritten, and the Rust allocator may not immediately reuse the freed pages. The bug could manifest only under specific timing conditions — for example, when system memory pressure causes the allocator to recycle pages more aggressively, or when the background thread is scheduled after a context switch that corrupts the stack.

Assumptions and Input Knowledge

To understand this message, the reader needs substantial background knowledge. First, the architecture of the SUPRASEAL_C2 pipeline: that Groth16 proofs involve multiple partitions, each with synthesis (circuit evaluation) producing A/B/C polynomials, followed by GPU MSMs and NTTs. Second, the FFI boundary between Rust and C++: Rust builds a Vec<Assignment<S>> and passes its buffer pointer to C++, which treats it as a C array. Third, the semantics of C++ lambda captures: [&] captures by reference, and capturing a pointer parameter by reference means capturing the stack variable holding the pointer, not the pointer value. Fourth, the concept of a split API: start returns while background work continues, requiring careful lifetime management.

The assistant's reasoning also relies on knowledge of the specific codebase: the groth16_pending_proof struct, the prep_msm_thread field, the generate_groth16_proofs_start_c and finalize_groth16_proof_c functions, and the Rust PendingProofHandle type. The assistant has been working with this code for many rounds and has built a detailed mental model of the call chain.

Output Knowledge Created

This message creates critical knowledge: the confirmation that the Phase 12 split API has a use-after-free bug in the provers array lifetime. This knowledge directly informs the fix: copying the provers array into the heap-allocated pending proof struct. It also informs the broader understanding that the split API's memory safety depends on careful lifetime management across the Rust/C++ boundary — a lesson that applies to any asynchronous FFI design.

The message also reinforces the importance of the assistant's investigative methodology: tracing memory ownership through the call chain, examining both the Rust and C++ sides, and verifying assumptions by reading the actual code rather than relying on mental models. The assistant's suspicion about memory accumulation led to the deeper discovery of a concurrency bug — a reminder that performance investigations often uncover correctness issues.

The Broader Significance

This message exemplifies a class of bugs that are becoming increasingly common as systems programmers adopt asynchronous, split-phase APIs for GPU acceleration. The tension between performance (returning early to overlap work) and safety (ensuring all data lives long enough for background threads) is a fundamental challenge in heterogeneous computing. The Rust/C++ FFI boundary is particularly treacherous because Rust's ownership model (which would prevent this bug at compile time) is invisible to the C++ compiler.

The fix — copying the provers array into the heap-allocated pending proof struct — is conceptually simple but has performance implications. Each copy adds latency and memory pressure. The assistant's subsequent work on memory optimization (early deallocation of NTT evaluation vectors, buffer tracking with atomic counters, and synthesis/GPU channel capacity tuning) represents the ongoing effort to balance these competing constraints.

In the end, the Phase 12 split API delivered a 2.4% throughput improvement while fixing a critical concurrency bug. But the journey to that result — the hours of tracing, the careful reading of C++ lambda captures, the realization that a "working" system was reading freed memory — is a testament to the complexity of modern GPU-accelerated cryptography and the rigor required to build it safely.