The Config That Unlocked the GPU: Understanding Phase 8's Final Piece

The Message

[assistant] [edit] /home/theuser/curio/extern/cuzk/cuzk-core/src/config.rs Edit applied successfully.

At first glance, this message from the opencode coding session appears almost trivial — a single-line confirmation that an edit to a configuration file was applied. Yet this brief message, <msg id=2197>, represents the culmination of a sophisticated multi-file refactoring effort spanning seven source files and nearly two hundred lines of code. It is the moment when Phase 8 of the cuzk SNARK proving engine — the Dual-Worker GPU Interlock — became a configurable, deployable reality.

Context: The Problem Phase 8 Solved

To understand why this message matters, one must understand the architectural problem it solved. The cuzk proving engine generates Groth16 proofs for Filecoin's Proof-of-Replication (PoRep) protocol, a computationally intensive process that involves both CPU-bound circuit synthesis and GPU-bound number-theoretic transforms (NTT) and multi-scalar multiplications (MSM). Earlier phases of optimization had revealed a critical bottleneck: GPU utilization was far from optimal, with idle gaps between partition computations where the GPU sat idle while the CPU prepared the next batch of work.

Phase 7 had introduced per-partition dispatch, which improved GPU utilization by dispatching individual partitions rather than waiting for all partitions to be synthesized. However, benchmarking revealed that a static mutex in the C++ CUDA kernel (generate_groth16_proofs_c) was causing unnecessary serialization. This mutex, originally placed to protect shared state during GPU kernel execution, was locking out CPU-side preprocessing work that could safely run in parallel. The result was a structural GPU idle gap — one worker would hold the mutex while running CUDA kernels, and the other would be blocked from even starting its CPU preprocessing.

Phase 8's insight was elegant: narrow the mutex scope to cover only the CUDA kernel region (NTT+MSM, batch additions, and tail MSMs), allowing CPU preprocessing and the b_g2_msm computation to run outside the lock. This meant two GPU workers per device could interleave — one running CUDA kernels while the other performed CPU work, and vice versa. The dual-worker interlock pattern effectively doubled GPU utilization without requiring additional hardware.

The Seven-File Journey

The implementation touched every layer of the proving stack, each file playing a specific role in threading the mutex pointer from C++ all the way up to the engine configuration:

  1. groth16_cuda.cu — The C++ CUDA kernel was refactored to remove the static mutex and accept a passed-in mutex pointer parameter, with the lock scope narrowed to only the critical GPU kernel region.
  2. supraseal-c2/src/lib.rs — The FFI (Foreign Function Interface) layer was updated to declare the new mutex pointer parameter in the extern declaration and wrapper, along with new alloc_gpu_mutex and destroy_gpu_mutex helper functions that create C++ std::mutex instances on the Rust side.
  3. bellperson — The Rust bellperson library, which provides the Groth16 proving primitives, received a gpu_mutex parameter added to prove_from_assignments and create_proof_batch_priority_inner.
  4. pipeline.rs — The gpu_prove function signature was updated to accept an optional GpuMutexPtr, with non-engine callers passing std::ptr::null_mut().
  5. engine.rs — The engine, the largest change, was restructured to create one C++ mutex per GPU device, spawn gpu_workers_per_device workers per GPU (defaulting to 2), and pass the shared mutex pointer to each worker's gpu_prove call.
  6. config.rs — The configuration struct gained the gpu_workers_per_device field.
  7. cuzk.example.toml — The example configuration file was updated to document the new parameter. The subject message <msg id=2197> is the confirmation of step 6 — the config change. It is the last code change before the example config documentation, and it represents the moment when the entire Phase 8 architecture became controllable through configuration rather than requiring code changes.

Why This Message Was Written

The assistant wrote this message as part of a methodical, step-by-step implementation process. The surrounding messages reveal a clear pattern: the assistant maintained a todo list tracking each sub-task, worked through the layers from bottom (C++) to top (configuration), and verified each step before proceeding. The config change was deliberately placed last because it depends on all the infrastructure being in place — the engine needed to know how many workers to spawn, which required the mutex plumbing to exist, which required the C++ kernel to accept the mutex pointer.

The reasoning visible in the preceding messages shows careful consideration of cross-language concerns. When the assistant reached the engine changes in <msg id=2181>, it paused to reconsider a critical design decision:

"Wait — I need to reconsider. The C++ side expects a std::mutex* (C++ std::mutex). A Rust std::sync::Mutex<()> is NOT the same thing. I need to allocate a C++ std::mutex from the Rust side."

This moment of reflection reveals the assistant's awareness of the subtle pitfalls in cross-language FFI. A Rust Mutex and a C++ std::mutex have completely different memory layouts; passing one where the other is expected would cause undefined behavior. The assistant correctly rejected the fragile approach of using Box::new(0u8) as opaque storage and instead added proper C++ helper functions (create_gpu_mutex/destroy_gpu_mutex) to allocate and deallocate std::mutex instances on the C++ side, returning opaque pointers to Rust.

Assumptions and Design Decisions

The config change embeds several assumptions worth examining:

Default value of 2: The assistant chose gpu_workers_per_device = 2 as the default, based on the dual-worker interlock design. This assumes that two workers per GPU is the optimal number — enough to keep the GPU busy while avoiding excessive CPU contention. Later benchmarking would validate this assumption, showing that 2 workers per device achieved 100% GPU efficiency for single-proof workloads.

The parameter name: gpu_workers_per_device was chosen to be explicit about what it controls — the number of worker threads per physical GPU device. This avoids ambiguity with partition_workers (which controls how many CPU threads participate in synthesis) and makes the configuration self-documenting.

Configurability: Rather than hardcoding the dual-worker behavior, the assistant made it configurable. This allows operators to tune the parameter for their specific hardware — machines with more CPU cores might benefit from more workers, while those with fewer cores might need fewer. It also provides a safety valve: if the dual-worker pattern causes issues, operators can fall back to 1 worker per device.

The PipelineConfig struct: The config change was placed in the existing PipelineConfig struct, which already contained pipeline-related parameters like partition_workers, synthesis_lookahead, and pipeline_mode. This was a natural home for the new parameter, consistent with the existing configuration architecture.

Input Knowledge Required

To understand this message, one needs knowledge of:

Output Knowledge Created

This message, combined with the preceding edits, created:

The Broader Significance

What makes this message noteworthy is not its content but its position in the larger narrative. It is the last code change of a major architectural refactoring — the moment when all the pieces finally connect. The C++ kernel has been refactored, the FFI layer updated, the Rust library modified, the pipeline function signature changed, the engine restructured, and now the configuration knob is in place. The next message updates the example config, and then the assistant moves on to building and benchmarking.

This pattern — bottom-up implementation culminating in a configuration change — is characteristic of well-structured software engineering. The assistant worked from the lowest layer (C++ kernel) upward through each abstraction boundary, ensuring that each layer could support the changes required by the layers above. The config change at the top is the simplest edit but depends on every layer below being correct.

The message also illustrates a key insight about optimization work: the most impactful changes often involve rethinking synchronization boundaries rather than rewriting algorithms. Phase 8's 13-17% throughput improvement came not from faster CUDA kernels but from smarter scheduling — letting two workers share a GPU by narrowing the mutex scope. The config parameter that controls this behavior is a small knob with outsized impact.

Conclusion

The single-line confirmation of a config file edit may seem unremarkable, but in the context of the Phase 8 Dual-Worker GPU Interlock implementation, it represents the completion of a carefully orchestrated, multi-layer refactoring. The assistant's methodical approach — working from the C++ kernel upward through FFI, Rust library, pipeline, engine, and finally configuration — ensured that each layer was properly prepared before the configuration knob was added. The gpu_workers_per_device parameter, born from a moment of careful reconsideration about C++ mutex layout compatibility, would go on to enable 100% GPU efficiency and double-digit throughput improvements in the cuzk SNARK proving engine.