The Ghost of a Double Allocation: A Case Study in Incremental Refactoring Under the Split API

In the midst of a complex, multi-layered refactoring to implement Phase 12's split API for the cuzk SNARK proving engine, the assistant issued a brief but revealing message:

I see — the old code still has the auto* pp = new which is wrong now since pp was allocated earlier. Let me replace this section: [edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu Edit applied successfully.

At first glance, this appears to be a trivial fix: remove a stale new expression that should have been deleted during an earlier edit. But this message captures a pivotal moment in a much larger architectural transformation — the decoupling of the GPU worker's critical path from CPU post-processing in the Groth16 proof generation pipeline. To understand why this one-line observation matters, we must trace the reasoning that led to it and the design pressures that made it inevitable.

The Split API: A Design Born from Memory Bandwidth Analysis

The Phase 12 split API did not emerge from abstract architectural ambition. It was the direct consequence of a deep, data-driven investigation into GPU utilization patterns. Throughout Phases 8 through 11, the assistant and user had been systematically profiling the cuzk proving engine, using TIMELINE analysis to identify why GPU utilization dipped. Phase 9's PCIe transfer optimization had improved single-worker throughput by 14.2%, but dual-worker mode revealed PCIe bandwidth contention. Phase 10's two-lock design was abandoned after discovering fundamental CUDA device-global synchronization conflicts. Phase 11 identified DDR5 memory bandwidth contention as the primary bottleneck, with three targeted interventions yielding a 3.4% improvement at best.

The critical insight that motivated Phase 12 was this: the b_g2_msm computation (~1.7 seconds with 32 threads) runs after the GPU lock is released, yet it still blocks the GPU worker from picking up the next synthesis job. The GPU worker's critical path — acquire lock, launch kernels, release lock — was being serialized with CPU post-processing that had no dependency on the GPU lock. The split API was designed to hide this latency by allowing the GPU worker to loop back and acquire the next job immediately after releasing the GPU lock, while a separate thread handles the b_g2_msm finalization.

The Handle Allocation Problem

The core technical challenge of the split API was one of memory lifetime management. The existing generate_groth16_proofs_c function was a monolithic ~1100-line behemoth that allocated all its state (vectors, MSM results, batch-add results, split flags, timing data) as local variables on the stack. These locals were captured by reference in the worker threads — both the GPU threads (per_gpu) and the prep_msm_thread that runs b_g2_msm. In the original design, this was safe because the function didn't return until all threads had joined.

The split API required a fundamentally different lifetime model. The function needed to return a handle before prep_msm_thread had finished, meaning the shared state had to outlive the function call. The solution was to heap-allocate a groth16_pending_proof struct that would serve as the stable memory home for all shared state, with the function's local variables becoming aliases to the handle's fields.

The assistant's initial approach (in [msg 2863]) was to allocate the handle at the end of the function, just before returning, and move the local state into it. But this ran into a critical problem: prep_msm_thread was still running, holding references to the local results and batch_add_res. Moving them into the handle would leave dangling references. As the assistant realized in [msg 2864], "The solution: allocate the handle early (before starting threads), put results and batch_add_res in it from the start."

This triggered a cascade of edits across messages 2866 through 2877. The assistant progressively moved each piece of shared state into the handle: first results and batch_add_res, then split_vectors_l/a/b and tail_msm_*_bases, then the l_split_msm/a_split_msm/b_split_msm flags, then caught_exception. Each edit required careful attention to the order of declarations and the aliasing pattern, because the threads captured these variables by reference and the references had to remain valid throughout execution.

The Ghost Allocation

It was in this context — after seven rounds of incremental restructuring — that the assistant discovered the stale auto* pp = new in message 2878. The original code had allocated the pending handle at the end of the function, just before the epilogue. But the assistant had already moved the allocation to the beginning of the function (in [msg 2866]), making the old new a duplicate that would silently overwrite the handle pointer and leak the earlier allocation.

This is a classic refactoring hazard. When you restructure code incrementally — moving an allocation earlier, changing initialization order, aliasing fields — the old code paths can survive as dead branches that compile but produce wrong behavior. The assistant caught this one before building, which saved what would likely have been a confusing debugging session involving memory corruption or mysterious segfaults.

What makes this message interesting is not the fix itself, but what it reveals about the assistant's mental model. The assistant was holding a complex, multi-step refactoring in working memory: the handle struct definition, the early allocation, the field aliases, the thread capture patterns, the split entry points, and the finalization function. Within that mental model, the assistant recognized that the old new was a relic — a ghost from the previous design that had not been cleaned up during the incremental edits. The observation "the old code still has the auto* pp = new which is wrong now since pp was allocated earlier" demonstrates a moment of cognitive coherence: the assistant traced the allocation through the restructured code, identified the inconsistency, and corrected it.

Assumptions and Their Validity

The assistant made several assumptions during this refactoring, most of which were correct but one of which deserves scrutiny. The core assumption was that moving shared state into a heap-allocated handle, with local aliases, would preserve the validity of thread references. This is correct in C++ as long as the handle outlives all threads — which it does, because the handle is either stored in the pending handle return path (for the split API) or deleted after prep_msm_thread.join() (for the synchronous path).

A more subtle assumption was that the split_vectors and tail_msm_bases vectors could be moved into the handle after construction but before the threads started. In [msg 2868], the assistant initially tried to allocate the handle after constructing these vectors and move them in, then realized the threads captured them by reference and needed stable addresses. The solution was to allocate the handle first and construct the vectors directly in the handle's fields. This was the right call, but it required an additional edit that the assistant caught mid-stream.

The Broader Significance

This message, for all its brevity, is a microcosm of the entire Phase 12 effort. The split API was a pragmatic response to a bottleneck identified through rigorous profiling — not a speculative optimization but a targeted intervention designed to hide a specific 1.7-second latency. The refactoring was carried out incrementally, with each edit building on the previous one, and with the assistant maintaining awareness of the entire state graph (what lives where, what references what, what outlives what). The discovery of the stale new was a natural consequence of this incremental approach: when you move pieces around, you sometimes leave old scaffolding behind.

The article could end here, but the deeper lesson is about the nature of cross-language FFI refactoring in performance-critical systems. The split API required coordinated changes across C++ struct definitions, CUDA kernel entry points, Rust FFI declarations in lib.rs, Rust wrapper functions in bellperson/supraseal.rs, pipeline integration in pipeline.rs, and engine worker loop restructuring in engine.rs. A single logical change — "decouple GPU work from CPU post-processing" — propagated through four layers of abstraction and two programming languages. The C++ handle allocation was just one node in this dependency graph, but if it had been wrong (double allocation, dangling reference, memory leak), the entire split API would have been compromised.

In the end, the assistant's observation was not just about removing a new statement. It was about maintaining the integrity of a complex state graph under active transformation — and catching the inconsistencies before they became bugs.