The Split API Design: Decoupling b_g2_msm from the GPU Worker's Critical Path

In the relentless pursuit of throughput for Filecoin's SUPRASEAL_C2 Groth16 proof generation pipeline, every millisecond matters. Message [msg 2849] represents a pivotal design moment in the optimization journey — the point where the assistant, having exhaustively characterized memory bandwidth bottlenecks in Phase 11, turns its attention to a new class of latency: the CPU-bound b_g2_msm computation that blocks the GPU worker from picking up its next job. This message is the design document for Phase 12's split API, a surgical intervention to hide ~1.7 seconds of post-GPU processing by restructuring the C++/Rust FFI boundary.

The Context: Why This Message Exists

To understand why this message was written, one must trace the optimization arc that preceded it. The assistant had just completed Phase 11, a set of three memory-bandwidth interventions targeting TLB shootdowns, L3 thrashing, and DDR5 contention. The best result was a 3.4% improvement (36.7 s/proof vs. 38.0 s baseline), achieved by reducing groth16_pool from 192 to 32 threads. But the user, unsatisfied with incremental gains, posed a direct question: could b_g2_msm — a ~1.7-second CPU multi-scalar multiplication on the G2 curve — be shipped to a separate thread to unblock the GPU worker more quickly?

This question landed in a mind already steeped in the pipeline's internals. The assistant had spent the previous messages ([msg 2834] through [msg 2840]) tracing the dependency chain inside generate_groth16_proofs_c, the monolithic C++ function that orchestrates the entire proof generation. It had discovered a crucial fact: b_g2_msm runs after the GPU mutex is released but before the function returns. The GPU worker in engine.rs calls this function synchronously, meaning it cannot loop back to pick up the next synthesized partition until b_g2_msm and the epilogue (final point additions that assemble the proof) complete. With per-partition proving (one circuit per call), the GPU kernel takes ~1.8 seconds, b_g2_msm takes ~1.7 seconds, and the epilogue is negligible (~1 ms). The worker's cycle is: acquire lock → GPU kernels → unlock → b_g2_msm + epilogue → loop. The ~1.7 seconds of b_g2_msm is dead time from the worker's perspective — it holds no GPU resources, yet it cannot proceed.

Timing data from the Phase 11 benchmark confirmed the problem. The assistant had extracted logs showing gpu_total_ms (the GPU kernel portion) ranging from 1,348 ms to 5,814 ms, while b_g2_msm_ms ranged from 1,521 ms to 4,569 ms. In many cases, b_g2_msm finished after the GPU kernels, meaning the worker was literally waiting for CPU computation while the GPU sat idle. The cold-start partition showed gpu_total_ms=5,814 vs. b_g2_msm_ms=4,569 (GPU was slower), but subsequent partitions showed the reverse: gpu_total_ms=1,348 vs. b_g2_msm_ms=1,521 — a 173 ms stall, or gpu_total_ms=1,759 vs. b_g2_msm_ms=2,069 — a 310 ms stall. These gaps, while individually small, compound across ten partitions per proof, and they represent a structural inefficiency: CPU post-processing is serialized on the GPU worker's critical path.

The Design Space: Three Approaches Considered

Message [msg 2849] opens with a meticulous catalog of the epilogue's dependencies. The assistant reads lines 990-1037 of groth16_cuda.cu and enumerates exactly what data the final proof assembly needs:

GPU results: results.h[c], results.l[c], results.a[c], results.b_g1[c] — from batch addition and MSM kernels
b_g2_msm result: results.b_g2[c] — from the CPU multi-scalar multiplication
Batch addition results: batch_add_res.l[c], .a[c], .b_g1[c], .b_g2[c] — from GPU batch addition
Input randomness: r_s[c], s_s[c] — the random scalars for the proof
Verifying key: vk->delta_g1, vk->alpha_g1, vk->beta_g1, vk->delta_g2, vk->beta_g2 — from the SRS (Structured Reference String)
Split flags: l_split_msm, a_split_msm, b_split_msm — computed during prep_msm This dependency analysis is the foundation for all subsequent design reasoning. The assistant recognizes that the epilogue is tightly coupled: it needs both GPU outputs (available after GPU unlock) and the b_g2_msm result (which may still be in flight). Any split design must preserve this coupling while allowing the GPU worker to escape. The assistant then walks through three approaches, each with different trade-offs: Approach 1: Fire-and-forget in C++. Spawn a detached thread for b_g2_msm + epilogue inside the C++ function, return the proof via an atomic flag and shared buffer. This minimizes Rust changes but requires the C++ function to manage thread lifetime and signal completion. The assistant quickly dismisses this because the function still blocks until the proof is ready — the caller gets no benefit. Approach 2: Return intermediates to Rust. Have C++ return the MSM results without running b_g2_msm, then let Rust orchestrate the final assembly. This is architecturally clean but requires a major C ABI change — the function signature must change, and Rust must handle G2 curve arithmetic it doesn't currently own. Approach 3: The pending-proof handle (chosen). Split generate_groth16_proofs_c into two FFI calls: generate_groth16_proofs_start_c (returns an opaque handle after GPU unlock) and finalize_groth16_proof (joins the b_g2_msm thread, runs the epilogue, writes the proof). The Rust GPU worker calls the first function, sends the handle to a finalizer thread, and immediately loops back for the next job. This is the sweet spot: minimal C ABI change (just a new handle type), clean separation of concerns, and no thread management complexity in Rust (the C++ side still owns the b_g2_msm thread).

The Reasoning: Why This Design Wins

The assistant's choice reflects a deep understanding of the system's constraints. The critical insight is stated explicitly: "we don't need to split the C API. We just need the engine's GPU worker to not wait for b_g2_msm." The C++ function's internal structure is fine — the problem is that the Rust worker calls it synchronously. By introducing a two-phase API, the assistant decouples the caller's blocking behavior from the function's internal parallelism.

The design preserves several important properties:

Memory stability: The handle struct in C++ holds all the state needed for finalization. By allocating it early (before GPU work begins), the assistant ensures that memory addresses referenced by GPU kernels remain valid even after the "start" function returns. This is critical because the GPU kernels may still be writing to device buffers that the handle's pointers reference.
Thread safety: The b_g2_msm thread is spawned inside C++ and joined during finalization. The Rust side never touches it directly. This avoids cross-language thread management issues and keeps the FFI boundary simple.
Error handling: The existing RustError return type is preserved. If the GPU kernel fails, the start function returns an error before any handle is created. If b_g2_msm fails, the finalize function returns the error.
Minimal Rust refactoring: The GPU worker loop changes from a single synchronous call to a two-step pattern, but the result-processing logic (tracker updates, partition assembly, error handling) is preserved. The assistant later designs helper functions (process_partition_result, process_monolithic_result) to encapsulate this logic and avoid code duplication.

Assumptions and Potential Blind Spots

The design makes several assumptions worth examining:

That b_g2_msm is worth hiding. The assistant's timing analysis shows stalls of 173-310 ms per partition, but the overall throughput impact depends on whether the GPU worker would otherwise be waiting for synthesis jobs. If the synthesis pipeline produces partitions slower than the GPU worker consumes them, hiding b_g2_msm gains nothing — the worker would idle anyway. The assistant implicitly assumes that synthesis is not the bottleneck, or that the pipeline is balanced such that any reduction in GPU worker cycle time translates to throughput. The later benchmarking of Phase 12 would validate or refute this assumption.

That the handle's memory is safe. The groth16_pending_proof struct must be allocated with a stable address before GPU kernels start, because those kernels write to buffers that the handle references. If the handle is allocated on the stack or in a temporary that gets moved, dangling pointers result. The assistant recognizes this risk and designs for early allocation.

That the two-phase API doesn't introduce excessive complexity. The Rust engine worker loop, already complex with error handling, tracker updates, and partition assembly, now must manage pending handles and finalization tasks. The assistant mitigates this with helper functions, but the risk of subtle bugs (e.g., forgetting to finalize a handle on error) is real.

That the GPU unlock point is the right boundary. The "start" function returns after GPU unlock, but before b_g2_msm completes. This means the handle's b_g2 field is not yet populated. The finalize function must handle the case where b_g2_msm is still running (join the thread) or has already completed (fast path). The assistant doesn't explicitly discuss this race condition, but the thread join semantics handle it naturally.

The Thinking Process: A Window into Iterative Design

What makes this message remarkable is the visible thinking process. The assistant doesn't arrive at the split API immediately — it works through alternatives, discarding each for concrete reasons. The reasoning is structured as a series of refinements:

Dependency mapping: "The epilogue (lines 990-1037) needs..." — a precise enumeration of what data flows where.
First design thought: "The simplest split: instead of moving all of this to Rust, keep the epilogue in C++ but have the function return immediately after GPU unlock, with a handle to the pending b_g2_msm."
Self-correction: "Actually, even simpler — let me think about what the user asked." The assistant re-centers on the user's framing: unblock the GPU worker, not redesign the C API.
The cleanest approach crystallizes: A three-point plan emerges: (1) C++ returns opaque handle, (2) separate FFI call finalizes, (3) Rust worker calls first function, sends handle to finalizer thread, loops back.
Implementation begins: "Let me implement this. The handle struct in C++ will hold everything needed for finalization" — followed by a read command to examine the C++ source. This pattern — analyze, propose, correct, refine, implement — is characteristic of expert systems design. The assistant is not just writing code; it's navigating a complex dependency graph with real-world constraints (FFI boundaries, memory safety, thread lifetimes) and making architectural decisions that balance performance against maintainability.

Input and Output Knowledge

To understand this message, the reader needs:

Knowledge of the SUPRASEAL_C2 pipeline: That Groth16 proofs involve multiple MSM operations (multi-scalar multiplications on G1 and G2 curves), that proof generation is partitioned across 10 partitions, and that the pipeline has a synthesis phase (CPU) followed by a proving phase (GPU + CPU).
Understanding of the GPU worker architecture: That the engine spawns multiple GPU workers that compete for a mutex, each processing one partition at a time, and that the worker loop is synchronous.
Familiarity with the C++/Rust FFI boundary: That generate_groth16_proofs_c is a C-compatible function called from Rust via extern "C", and that changing its signature requires coordinated changes in both languages.
The timing data from Phase 11: That b_g2_msm takes ~1.7 seconds with 32 threads and often outlasts the GPU kernel time. The message creates new knowledge:
The split API design: A two-phase FFI contract that decouples GPU work from CPU post-processing.
The pending-proof handle concept: An opaque C++ struct that holds all state needed for finalization, allocated early for memory stability.
The dependency map: A precise catalog of what data the epilogue consumes and where each piece originates.
The architectural blueprint: How the Rust engine worker loop should be restructured to use the split API, with a finalizer thread handling proof completion.

Conclusion

Message [msg 2849] is a masterclass in systems-level optimization design. It demonstrates how to identify a structural inefficiency (CPU post-processing serialized on the GPU worker's critical path), analyze its root cause (the synchronous FFI call), and design a minimally invasive fix (the two-phase split API). The assistant's thinking process — cycling through alternatives, testing each against the system's constraints, and settling on the approach that maximizes benefit while minimizing risk — is the kind of reasoning that separates effective optimization from mere tinkering.

The split API would go on to be implemented across C++, Rust FFI, and the engine worker loop in the subsequent chunks of Segment 29. Its effectiveness would be measured in the benchmarks that followed, but the design itself — captured in this single message — represents the moment the assistant saw the path forward. In a pipeline already squeezed for every microsecond, shaving ~1.7 seconds off the GPU worker's cycle time was not just an optimization; it was a re-architecting of how the proving engine thinks about work.