The Bridge: Wiring the Split API Across the C++/Rust FFI Boundary

In the course of optimizing the SUPRASEAL_C2 Groth16 proof generation pipeline for Filecoin PoRep, a single brief message — <msg id=2888> — marks the moment when a deep architectural refactoring crosses the language boundary from C++ CUDA code into Rust. The message reads simply:

[edit] /home/theuser/curio/extern/supraseal-c2/src/lib.rs Edit applied successfully.

On its surface, this is one of the most unremarkable messages in the entire conversation: a tool call result confirming that an edit was applied. Yet this message is the critical bridge in the Phase 12 split API implementation, connecting the newly refactored C++ generate_groth16_proofs_start_c and finalize_groth16_proof functions to the Rust side of the proving pipeline. Without this FFI declaration, the entire preceding refactoring of groth16_cuda.cu — dozens of edits across hundreds of lines of complex CUDA code — would remain inaccessible to the Rust application that orchestrates proof generation.

The Context: Why This Message Exists

To understand why this message was written, one must trace the reasoning that led to it. The Phase 12 split API was born from a detailed bottleneck analysis performed during Phase 11, which identified DDR5 memory bandwidth contention as the primary performance limiter in dual-worker GPU mode. Within that analysis, a specific sub-bottleneck emerged: the b_g2_msm (B-G2 multi-scalar multiplication) computation, which takes approximately 1.7 seconds with 32 threads, runs after the GPU lock is released, yet still blocks the GPU worker from picking up the next synthesis job. The worker cannot proceed to the next proof while it is still running the CPU-side epilogue that includes b_g2_msm.

The insight was simple but powerful: if the GPU worker's critical path could be decoupled from the CPU post-processing, the worker could loop back immediately to begin the next proof's GPU work while b_g2_msm completes in the background. This is a classic latency-hiding pattern, analogous to prefetching or double-buffering in systems design. The split API design therefore called for two entry points: generate_groth16_proofs_start_c, which runs the GPU kernels, releases the GPU lock, and returns an opaque handle; and finalize_groth16_proof, which joins the deferred b_g2_msm thread, runs the proof epilogue, and writes the final proof.

The Refactoring That Preceded This Message

Messages <msg id=2859> through <msg id=2887> document a painstaking refactoring of the ~1100-line generate_groth16_proofs_c function in groth16_cuda.cu. The core challenge was that the function's local variables — results, batch_add_res, split_vectors_l/a/b, tail_msm_l/a/b/b_g2_bases, and the split-MSM boolean flags — were all captured by reference in the GPU worker threads and the prep_msm_thread. Moving these into a heap-allocated groth16_pending_proof struct required careful aliasing to ensure that all thread references remained valid throughout execution.

The assistant encountered and fixed two compilation errors during this refactoring. The first was an ordering issue: the split-MSM flag aliases referenced pp (the pending proof pointer) before it was allocated. The second was a subtle C++ type mismatch: a ternary expression between tail_msm_b_g2_bases.data() (non-const affine_fp2_t*) and points_b_g2.data() (const affine_fp2_t* from a slice_t) produced an ambiguous type that the mult_pippenger template could not resolve. Both issues were diagnosed through iterative build-test cycles — running cargo build --release -p cuzk-daemon, parsing the compiler errors, and applying targeted fixes.

By message <msg id=2886>, the C++ side compiled successfully. The assistant then stated the next step explicitly: "Now I need to add the Rust FFI declarations and wire up the split API. Let me update supraseal-c2/src/lib.rs."

What the Edit Contained

While the message text does not show the diff, the context makes clear what this edit to lib.rs contained. The file supraseal-c2/src/lib.rs is the Rust FFI boundary for the supraseal-c2 CUDA library — it declares extern "C" functions that Rust code can call into the C++ library. The edit would have added:

generate_groth16_proofs_start_c: The new entry point that takes the same parameters as the original generate_groth16_proofs_c plus an extra void** out_handle parameter for returning the pending proof handle.
finalize_groth16_proof: The new entry point that takes a void* handle (the opaque pending proof) and completes the proof generation, returning the final result.
PendingProofHandle: A Rust wrapper type (likely a newtype around *mut c_void or similar) that represents the opaque handle returned by the start function. These declarations are the essential glue that makes the split API callable from Rust. Without them, the C++ functions exist in isolation — compiled into the static library but unreachable from the application code that orchestrates proof generation.## Assumptions Embedded in the Edit Every FFI boundary crossing carries assumptions about type compatibility, memory layout, and lifetime semantics. This edit makes several notable assumptions: Memory stability: The groth16_pending_proof struct is heap-allocated in C++ and returned as an opaque void* pointer. The Rust side assumes that this pointer remains valid until finalize_groth16_proof is called and that the C++ side will handle all deallocation. This is a reasonable ownership pattern — C++ allocates, C++ frees — but it means Rust must not attempt to interpret or dereference the pointer. The opaque handle pattern deliberately prevents Rust from accessing the struct's internals, which is wise given the complexity of the C++ data structures involved. Thread safety: The split API assumes that finalize_groth16_proof can be called from a different thread than generate_groth16_proofs_start_c. The prep_msm_thread (which runs b_g2_msm) is std::thread-detached within the C++ code, and the Rust side is expected to join it indirectly through the finalize call. This means the Rust runtime must ensure that the finalize call happens on a thread that can block without starving the worker pool — hence the plan to spawn a separate tokio task for finalization. No concurrent access to the handle: The design assumes that the handle is used linearly — start, then finalize — with no concurrent calls on the same handle. This is enforced by the Rust worker loop's structure, but it is worth noting as an implicit contract.

Input Knowledge Required

To understand this message, one must grasp several layers of context:

The Groth16 proof pipeline: Knowledge that a Groth16 proof involves multiple MSM operations (multi-scalar multiplications), that b_g2_msm operates on the G2 curve (FP2 extension field), and that the epilogue includes verification equation computation and proof serialization.
The CUDA execution model: Understanding that GPU kernel launches are asynchronous, that cudaStreamSynchronize or similar barriers are needed before results are valid, and that GPU resources (device memory, streams, events) must be freed while holding the GPU mutex to prevent OOM in multi-worker scenarios.
The Rust FFI pattern: Familiarity with extern "C" declarations, the use of *mut c_void for opaque handles, and the RustError return type convention used throughout the supraseal-c2 library.
The worker loop architecture: Understanding that the cuzk-daemon engine spawns multiple GPU workers that compete for a GPU mutex, and that the worker loop currently blocks on the full proof generation before picking up the next synthesis job.
The Phase 11 bottleneck analysis: The discovery that DDR5 memory bandwidth contention between SpMV (sparse matrix-vector multiply) and Pippenger MSM bucket accumulation is the primary performance limiter, and that b_g2_msm contributes ~1.7 seconds of CPU-bound post-processing that could be overlapped with the next GPU kernel launch.

Output Knowledge Created

This message, combined with the subsequent edits to bellperson/src/groth16/prover/supraseal.rs and the engine worker loop, creates the following new knowledge:

The split API surface: A clear separation between GPU work (start) and CPU post-processing (finalize), enabling the worker to pipeline proof generation.
The opaque handle abstraction: A mechanism for passing complex C++ state across the FFI boundary without exposing internals to Rust, preserving encapsulation while enabling the deferred execution pattern.
A template for future split APIs: The pattern established here — allocate state early, alias thread-captured references into the handle, return opaque pointer, finalize later — can be applied to other GPU operations where CPU post-processing blocks the critical path.
A validated compilation path: The successful compilation of both C++ and Rust sides confirms that the type system constraints (const/non-const pointer compatibility, template argument deduction, FFI calling conventions) are satisfied.

The Thinking Process Visible in the Reasoning

The assistant's reasoning across messages <msg id=2859> through <msg id=2888> reveals a methodical, constraint-driven approach to a complex refactoring problem. The initial instinct was to duplicate the entire ~1100-line function — "the least-invasive approach" — but this was quickly rejected as too messy. The assistant then considered extracting a shared helper but recognized that "given the function's complexity and many local variables, extracting would be messy."

The chosen approach — modifying the existing function to accept an optional void** pending_out parameter — is a pragmatic compromise that minimizes code duplication while adding the new capability. The reasoning shows careful attention to the lifetime problem: "The results and batch_add_res are declared locally in the function and captured by reference in the threads. If I move them into the pending handle, the references from prep_msm_thread (still running) become dangling!"

This observation drove the decision to allocate the handle early and alias its fields as the local variables, ensuring that "the handle outlives all threads." The assistant's comment "All threads capture these references; the handle outlives all threads" (from the code comments at line 397-398) crystallizes the key insight: by allocating the handle on the heap at the top of the function and using references to its fields throughout, the data remains at stable addresses regardless of thread lifetimes.

The diagnostic process for the compilation errors is equally methodical. When the first build failed, the assistant ran grep to find the relevant lines, read the file to understand the ordering, and identified the root cause: "pp allocation is at line 378 but the split_msm flag aliases at line 367-372 reference pp before it's declared." For the mult_pippenger type error, the assistant traced the const/non-const pointer mismatch in the ternary expression and applied a cast to resolve it. Each error was addressed with a targeted edit, followed by a rebuild to verify the fix.

Broader Significance

This message, while brief, represents the moment when the Phase 12 split API transitions from a C++ implementation detail to a cross-language architectural change. The FFI boundary is often the hardest part of any multi-language system — it is where type systems collide, ownership models must be reconciled, and implicit assumptions become explicit bugs. The successful edit to lib.rs is the gate that the rest of the integration (the PendingProofHandle wrapper in supraseal.rs, the gpu_prove_start/gpu_prove_finish wrappers in pipeline.rs, and the worker loop restructuring in engine.rs) must pass through.

The article in [chunk 29.1] describes this as "the inherent complexity of cross-language FFI development, where a single logical change (splitting a function) requires coordinated modifications across C++ structs, Rust wrappers, and application-level orchestration." Message <msg id=2888> is the fulcrum of that coordination — the point where the C++ changes meet the Rust changes, and the entire split API becomes real.

In the broader narrative of the SUPRASEAL_C2 optimization effort, this message is a quiet but essential step. The Phase 12 split API, once fully integrated and benchmarked, has the potential to hide ~1.7 seconds of CPU post-processing per proof from the GPU worker's critical path, translating directly into throughput improvement. The edit to lib.rs is the bridge that makes that improvement possible.