The Split API: Decoupling GPU Kernel Execution from CPU Post-Processing in Groth16 Proof Generation

Introduction

In the relentless pursuit of throughput for Filecoin's Proof-of-Replication (PoRep) proving pipeline, every millisecond matters. The SUPRASEAL_C2 Groth16 proof generation system, a critical component of the Curio mining stack, had already undergone ten optimization phases targeting GPU utilization, PCIe transfer efficiency, and memory bandwidth contention. By Phase 11, the team had squeezed a 3.4% improvement from memory-bandwidth interventions, bringing proof time from 38.0 s to 36.7 s. But a persistent structural inefficiency remained: the GPU worker's critical path included approximately 1.7 s of CPU-bound b_g2_msm computation that ran after the GPU lock was released but before the worker could loop back to pick up the next synthesis job. Message 2855 marks the precise moment when the assistant transitioned from analyzing this bottleneck to implementing its solution — the Phase 12 split API.

The Message

The subject message is deceptively brief:

Now let me implement the C++ split. I'll add the groth16_pending_proof struct and two new FFI functions, then modify the existing function to return a pending handle when num_circuits == 1.

>

[edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu Edit applied successfully.

This single sentence, accompanied by a file edit, represents the culmination of an extensive design deliberation spanning messages 2839 through 2854. The assistant is announcing a commitment to implementation after having thoroughly analyzed the dependency chain, evaluated alternative approaches, and settled on a specific architectural strategy.

Context and Motivation

To understand why this message was written, one must trace the reasoning that preceded it. The Phase 11 benchmarks had revealed that DDR5 memory bandwidth contention was the primary bottleneck in dual-worker GPU mode. Three interventions were tested: serializing async_dealloc with a static mutex, reducing the groth16_pool thread count from 192 to 32, and adding a global atomic throttle flag. Intervention 2 alone delivered the best result — a 3.4% improvement — but the fundamental issue remained: the GPU worker's cycle included CPU post-processing that blocked it from accepting the next job.

The user then posed a pivotal question in message 2840: could b_g2_msm be shipped to a separate thread to unblock the GPU worker more quickly? This triggered a deep analysis of the per-partition proving flow. With num_circuits=1 (one partition per GPU call), the timing looked like this:

Design Deliberation

The assistant's reasoning process in the preceding messages reveals a careful evaluation of multiple design approaches. The first idea was a fire-and-forget approach in C++: spawn a detached thread for b_g2_msm + epilogue, return the proof via an atomic flag and shared buffer, and minimize Rust-side changes. The user selected this approach as "Recommended" in a question posed at message 2840, preferring cleaner separation of concerns.

However, as the assistant dug deeper into the implementation details, complications emerged. The r_s and s_s randomness pointers were borrowed from the Rust caller — if the GPU worker returned early, the Rust side would drop those buffers, leaving dangling references for the epilogue. The assistant considered two solutions: copying the 32-byte scalars into the handle (simple and safe) or keeping the Rust Vec alive through ownership transfer.

A more fundamental problem surfaced in message 2864: the results and batch_add_results structures were local variables captured by reference in the GPU threads and prep_msm_thread. Moving them into the pending handle after thread creation would invalidate those references. The solution was to allocate the groth16_pending_proof handle early — before any threads were spawned — and use its fields as the actual storage from the start. This ensured stable memory addresses throughout the function's lifetime and beyond, since the handle would outlive the function scope when returned to Rust.

Assumptions and Decisions

Several assumptions underpin this implementation. First, the assistant assumes that num_circuits == 1 (per-partition proving) is the dominant and most performance-critical case worth optimizing. The split API is designed with this assumption, and the existing synchronous function is preserved for backward compatibility. Second, the assistant assumes that the C++ side can safely allocate the handle on the heap and return an opaque pointer to Rust, with the Rust side responsible for calling the finalizer and destroying the handle. Third, the assistant assumes that the overhead of heap allocation and additional FFI crossings is negligible compared to the 1.7 s of b_g2_msm computation being hidden.

The decision to modify the existing function rather than creating a completely separate entry point reflects a pragmatic trade-off. The assistant initially considered duplicating the ~1100-line function but rejected that as too invasive. Instead, the chosen approach adds an optional void** pending_out parameter to the existing function — when non-null, the function skips the epilogue, packages state into a handle, and returns early. This minimizes code duplication while enabling the split behavior.

Input Knowledge Required

To fully understand this message, one needs substantial domain knowledge spanning multiple layers of the system. At the lowest level, one must understand the Groth16 proof generation algorithm and specifically the role of b_g2_msm — a multi-scalar multiplication in the G2 curve that computes the B component of the proof. At the system architecture level, one must understand the CUDA GPU programming model, the pinned memory transfer patterns, and the mutex-based GPU worker interlock that serializes access to the device. At the language boundary, one must understand the Rust FFI conventions, opaque pointer patterns, and ownership semantics across C++ and Rust. Finally, at the application level, one must understand the Curio engine's worker loop, the synthesis pipeline's partition production rate, and the tracker/result-processing machinery.

Output Knowledge Created

This message creates the foundation for the entire Phase 12 implementation. The groth16_pending_proof struct encapsulates all the state needed for deferred finalization: the b_g2_msm thread handle, the MSM results and batch addition results, the split MSM flags, the verifying key pointer, copies of the randomness scalars, and the deallocation data (split vectors and tail MSM bases). The two new FFI functions — generate_groth16_proofs_start_c and finalize_groth16_proof — define the contract between C++ and Rust for the split workflow.

This output directly enables the subsequent Rust-side changes: declaring the FFI functions in lib.rs, creating PendingProofHandle and prove_start/prove_finish wrappers in supraseal.rs, and ultimately restructuring the engine's GPU worker loop to call gpu_prove_start, spawn a tokio task for finalization, and immediately loop back for the next job. The split API is the linchpin that allows the GPU worker to decouple from CPU post-processing.

The Thinking Process

The assistant's reasoning visible in the surrounding messages reveals a methodical, iterative approach to system optimization. Rather than jumping to implementation, the assistant first maps the exact timing of the GPU worker cycle, identifies the 1.7 s b_g2_msm window, and verifies that offloading it would actually improve throughput (the answer: yes, by reducing the risk of GPU idle gaps when synthesis is slow). The assistant then traces the dependency chain through the C++ code, identifying every piece of state that the epilogue needs and ensuring it can be safely captured in the handle.

The most impressive aspect of the reasoning is the careful attention to memory safety across the FFI boundary. The assistant identifies the dangling pointer problem with r_s/s_s, the reference invalidation problem with results/batch_add_res, and the thread-safety implications of moving state after thread creation. Each problem is solved with a targeted intervention: copying scalars instead of borrowing, allocating the handle early, and aliasing local variables to handle fields. This systematic approach to correctness is essential when modifying performance-critical code where a single dangling pointer can cause silent corruption.

Conclusion

Message 2855 is a turning point in the optimization journey. After ten phases of incremental improvements targeting GPU utilization, PCIe bandwidth, and memory contention, the Phase 12 split API addresses a structural inefficiency in the worker architecture itself. By decoupling GPU kernel execution from CPU post-processing, the split API allows the GPU worker to loop back ~1.7 s sooner, reducing idle time and improving throughput. The message itself is brief — a single sentence announcing intent — but it represents the culmination of extensive analysis, the resolution of subtle memory-safety challenges, and the beginning of a significant refactoring across C++, Rust FFI, and application-level orchestration. The split API is not merely an optimization; it is a architectural change that redefines the contract between the GPU and CPU phases of proof generation.