Splitting the Monolith: Implementing Phase 12's Split API at the Rust-C++ FFI Boundary

Introduction

In the relentless pursuit of throughput optimization for Filecoin's Groth16 proof generation pipeline, the opencode session reaches a critical architectural inflection point. Message 2890 represents the moment when the assistant, having successfully implemented the C++ side of a "split API" design, turns its attention to the Rust FFI boundary — the bridge between high-level orchestration logic and low-level CUDA-accelerated computation. This message, though brief in its visible content, marks the transition from designing a latency-hiding strategy to actually implementing it across a complex cross-language codebase.

The subject of this article is a single assistant message at index 2890 in the conversation. In it, the assistant reads an existing Rust file to understand the prove_from_assignments function, with the explicit goal of creating a "split version." This seemingly simple act of reading a file is, in reality, the culmination of hours of prior analysis, debugging, and architectural design — and the beginning of one of the most intricate refactorings in the entire optimization campaign.

The Message: What It Says

The message is concise and direct:

Let me look at prove_from_assignments and then add the split version: [read] /home/theuser/curio/extern/bellperson/src/groth16/prover/supraseal.rs

The assistant then displays the contents of the file starting at line 270, showing the SendableGpuMutex struct and its documentation. The read operation reveals the existing Rust types that wrap C++ mutex pointers — the scaffolding upon which the split API must be built.

On its surface, this is a straightforward file read. But the context transforms it into something far more significant. The assistant is not casually browsing code; it is executing a carefully planned multi-step refactoring that spans C++ struct definitions, CUDA kernel orchestration, Rust FFI declarations, and application-level worker loop restructuring.

Why This Message Was Written: The Reasoning and Motivation

To understand why this message exists, one must trace the optimization journey that preceded it. The session had just completed Phase 11, a set of three memory-bandwidth interventions targeting DDR5 contention, TLB shootdowns, and L3 cache thrashing. The best result from Phase 11 was a 3.4% improvement — meaningful but not transformative. The bottleneck analysis revealed that the GPU worker's critical path was unnecessarily long because it included CPU post-processing work that could be deferred.

The specific culprit was b_g2_msm, a multi-scalar multiplication on the G2 curve that takes approximately 1.7 seconds with 32 threads. This computation runs after the GPU lock is released — meaning it doesn't block other GPU workers from acquiring the lock — but it does block the current worker from picking up the next synthesis job. The worker must wait for b_g2_msm to complete, then run the proof epilogue (final hashing, proof assembly), and only then can it loop back to process the next circuit batch.

The insight was elegant: if the worker could hand off the b_g2_msm and epilogue to a separate thread and immediately loop back to pick up the next job, throughput would increase by hiding this latency. This is the classic "split API" pattern — decompose a monolithic operation into a "start" that does the critical-path work and returns a handle, and a "finish" that completes the deferred work using that handle.

The user explicitly suggested this approach in the previous segment, asking whether b_g2_msm could be shipped to a separate thread. The assistant analyzed the dependency chain, confirmed feasibility, and designed the split API. By message 2890, the C++ implementation was complete and compiling. The next step was to wire it into the Rust layer.

How Decisions Were Made

The decision to read prove_from_assignments specifically reveals the assistant's design methodology. Rather than writing the split API from scratch, the assistant planned to use the existing monolithic function as a template — understanding its signature, its type parameters, its error handling, and its resource management patterns, then decomposing it into prove_start and prove_finish counterparts.

This is a deliberate architectural choice. The split API must preserve the exact same computation; it merely changes when parts of it execute. By starting from the existing function, the assistant minimizes the risk of introducing semantic differences between the monolithic and split paths. The SendableGpuMutex type shown in the read output is particularly relevant — the split API must handle mutex ownership and lifetime correctly across the two-phase protocol.

The assistant also decided to use an opaque handle (groth16_pending_proof*) returned from C++ to Rust, rather than marshaling all intermediate state across the FFI boundary. This avoids copying large vectors (results, batch_add_res, split vectors) and ensures memory stability. The handle is heap-allocated in C++, its fields serve as the shared state for both GPU threads and the deferred finalization, and it is freed when finalize_groth16_proof completes.

Assumptions Made

Several assumptions underpin this message and the broader Phase 12 design:

First, the assistant assumes that the b_g2_msm computation is genuinely independent of the GPU worker's next job. If there were hidden dependencies — for example, if b_g2_msm implicitly used GPU resources that the next job needs — the split would not yield the expected benefit. The Phase 11 analysis confirmed that b_g2_msm runs entirely on CPU (using the Pippenger MSM algorithm on G2 points), so this assumption is well-founded.

Second, the assistant assumes that spawning a separate thread for finalization will not cause resource contention that negates the latency hiding. The finalization thread consumes CPU cores and memory bandwidth, potentially competing with the next GPU worker's synthesis or SpMV operations. This is a real risk that can only be validated through benchmarking.

Third, the assistant assumes that the opaque handle approach works correctly across the FFI boundary. The handle is a groth16_pending_proof* allocated with new in C++ and passed as a void* through the C FFI. Rust receives it as an opaque pointer and must pass it back to C++ for finalization. Any mismatch in alignment, lifetime, or ownership would cause undefined behavior.

Fourth, the assistant assumes that the existing prove_from_assignments function's structure can be cleanly split. In practice, this function interleaves GPU work (kernel launches, MSM computations) with CPU work (SpMV, b_g2_msm, epilogue) in a way that may not partition neatly. The assistant's C++ refactoring (messages 2861-2886) addressed this by allocating the pending handle early and aliasing all shared state into it, ensuring that the split is mechanically sound even if the original code was not designed for it.

Input Knowledge Required

To fully understand this message, one needs knowledge spanning multiple domains:

Groth16 proof generation: The reader must understand that a Groth16 proof involves multiple multi-scalar multiplications (MSMs) on different curves (G1 and G2), a proof synthesis step (computing a/b/c vectors via sparse matrix-vector product, SpMV), and an epilogue that combines results into the final proof format. The b_g2_msm is specifically the MSM on the G2 curve for the B component of the proof.

CUDA GPU worker architecture: The proving engine uses multiple GPU workers that compete for a mutex-protected GPU. Each worker acquires the lock, runs CUDA kernels, releases the lock, then does CPU post-processing. The split API aims to move CPU post-processing out of the worker's critical path.

Rust FFI with C++: The codebase uses extern "C" functions to bridge Rust and C++. Opaque pointers (void*) are used to pass C++ objects across the boundary. The SendableGpuMutex type wraps a GpuMutexPtr (a *mut c_void) and implements Send and Sync to allow sharing across threads.

The optimization history: This message is meaningless without understanding that it is Phase 12 of a multi-phase optimization campaign. Phases 8-11 progressively identified and addressed GPU lock contention, PCIe transfer bottlenecks, DDR5 memory bandwidth contention, and now the CPU post-processing latency.

Output Knowledge Created

This message creates knowledge in several forms:

Immediate knowledge: The assistant learns the exact structure of prove_from_assignments — its type parameters, its return type, its use of SendableGpuMutex, and the surrounding module exports. This knowledge directly informs the design of prove_start and prove_finish.

Architectural knowledge: The split API design itself is a form of knowledge — the insight that b_g2_msm and the epilogue can be deferred without affecting correctness, and that this deferral can be achieved through a handle-based two-phase protocol.

Documentation knowledge: The subsequent messages (2891 onwards) will add PendingProofHandle, prove_start, and prove_finish to the Rust codebase, creating a permanent record of the split API design. This becomes part of the codebase's shared knowledge.

Performance knowledge: Once implemented and benchmarked, the split API will produce quantitative knowledge about whether latency hiding actually improves throughput, and by how much. This validates or refutes the assumptions underlying Phase 12.

The Thinking Process

The assistant's thinking process in this message is visible primarily through what it doesn't say — the implicit reasoning that leads from "C++ compiles" to "now I need to look at prove_from_assignments."

The assistant has just finished a grueling debugging session in C++ (messages 2861-2886), where it discovered and fixed two compilation errors: an ordering issue where pp was referenced before allocation, and a mult_pippenger type mismatch caused by const/non-const pointer ambiguity in a ternary expression. These are the kinds of subtle bugs that plague cross-language FFI development — a reference that looks correct in C++ but fails because the type system treats const and non-const pointers differently in template argument deduction.

Having fixed these issues and confirmed that cargo build --release -p cuzk-daemon succeeds, the assistant now faces the next layer of the onion: the Rust FFI declarations and wrappers. Message 2888 already added the extern "C" declarations for generate_groth16_proofs_start_c and finalize_groth16_proof to lib.rs. Now the assistant needs to create the Rust-side types and functions that call these C functions.

The assistant's choice to read prove_from_assignments reveals a methodical, template-based approach to refactoring. Rather than designing the split API in the abstract and then implementing it, the assistant anchors on the existing concrete implementation and asks: "What does this function do? How can I split it into start/finish while preserving its behavior?" This is a pragmatic engineering mindset — minimize risk by reusing proven patterns.

The read also serves a reconnaissance purpose. The assistant needs to understand the module's export structure to know where to add the new types and functions. The grep output from message 2889 shows pub(super) fn create_proof_batch_priority_inner, pub struct SynthesisCapacityHint, pub fn synthesize_circuits_batch, etc. — these are the existing public API that the split version must integrate with.

Broader Significance

Message 2890, for all its brevity, is a microcosm of the entire optimization campaign. It represents the moment when a theoretical insight (latency hiding through split API) meets the messy reality of cross-language FFI implementation. The assistant must navigate C++ struct definitions, CUDA thread safety, Rust type system constraints, and application-level orchestration — all while preserving correctness and avoiding regressions.

The split API also represents a philosophical shift in the optimization approach. Earlier phases focused on reducing resource contention (GPU lock narrowing, PCIe optimization, memory bandwidth reduction). Phase 12 takes a different tack: instead of reducing the cost of existing work, it restructures when work happens to hide latency. This is a higher-risk, higher-reward strategy — it doesn't make any individual operation faster, but it can improve throughput by keeping the GPU worker busy more of the time.

The success of this approach depends on the validity of the assumptions identified above, and on the assistant's ability to implement the split correctly across the C++/Rust boundary. The subsequent messages in the session will reveal whether the benchmark results validate the design, or whether unforeseen interactions (thread contention, memory pressure, FFI overhead) undermine the expected gains.

Conclusion

Message 2890 is a bridge — between C++ and Rust, between design and implementation, between the old monolithic API and the new split API. It is a message about reading code, but the reading is in service of writing, and the writing is in service of a carefully reasoned optimization strategy. The assistant's methodical approach — fix C++ compilation, add FFI declarations, study existing Rust code, implement split wrappers, integrate into engine — demonstrates the disciplined engineering required to optimize a complex, production-grade proving system. Whether Phase 12 succeeds or fails, the reasoning captured in this message and its surrounding context provides a valuable case study in latency hiding, FFI design, and performance engineering at scale.