The Architecture of a Cross-Crate Dependency: How One Grep Shaped Phase 8's Dual-Worker GPU Interlock

In the midst of implementing Phase 8 of the cuzk SNARK proving engine — a dual-worker GPU interlock designed to eliminate idle GPU gaps by narrowing a C++ static mutex — the assistant encountered a subtle architectural constraint that threatened to derail the entire effort. Message [msg 2202] captures the precise moment of recognition: the engine module cannot directly access the supraseal_c2 crate, yet it needs to call alloc_gpu_mutex() and destroy_gpu_mutex() to create per-GPU C++ mutexes for the dual-worker scheme. This message, though brief, reveals a critical insight about Rust module dependency architecture and demonstrates how a single grep command can illuminate the path forward.

The Problem: A Dependency Chain with a Missing Link

The Phase 8 design called for a fundamental change to how GPU workers interact with the CUDA kernel region. Previously, a single static std::mutex in the C++ code (groth16_cuda.cu) guarded the entire generate_groth16_proofs_c function, creating a coarse lock that serialized all GPU work. The Phase 8 insight was to narrow this mutex to cover only the CUDA kernel region (NTT+MSM, batch additions, tail MSMs), allowing CPU preprocessing and b_g2_msm to run outside the lock. This meant two GPU workers per device could interleave — one doing CPU work while the other ran CUDA kernels.

The implementation spanned seven files and roughly 195 lines of changes across C++, Rust FFI, and the engine layer. By message [msg 2202], the assistant had already completed the C++ mutex refactor, the FFI plumbing in supraseal-c2/src/lib.rs, the Rust bellperson parameter additions, and the gpu_prove signature changes in pipeline.rs. The next step — Step 5 — was the engine changes in cuzk-core/src/engine.rs, where the actual worker spawning logic lives.

The engine needed to create one C++ std::mutex per GPU device and pass it to each spawned worker. The natural function to call was supraseal_c2::alloc_gpu_mutex(), a helper the assistant had just added to the FFI layer in [msg 2181]. But there was a problem: the engine module, living in cuzk-core, does not have supraseal_c2 as a direct dependency.

The Dependency Graph

Understanding why this matters requires tracing the crate dependency chain. The cuzk-daemon binary depends on cuzk-core, which contains the engine. cuzk-core depends on bellperson, a Groth16 proving library. And bellperson depends on supraseal-c2, the low-level C++/CUDA binding crate that provides the actual GPU proof generation functions. The chain looks like this:

cuzk-daemon → cuzk-core (engine) → bellperson → supraseal-c2

The engine can call functions from bellperson and from its own pipeline module, but it cannot directly call supraseal_c2::alloc_gpu_mutex() because supraseal_c2 is not in its dependency list. Adding supraseal_c2 as a direct dependency of cuzk-core would be possible but architecturally undesirable — it would create a tighter coupling and duplicate the dependency path that already exists through bellperson.

The Investigation: One Grep to Rule Them All

Message [msg 2202] opens with the assistant articulating this realization:

The engine can't directly access supraseal_c2 — it needs to go through bellperson or pipeline. Let me re-export the mutex creation functions through bellperson and then through the pipeline module.

The assistant then runs a grep for supraseal_c2 across the bellperson source tree, producing five matches. The results confirm that bellperson already has a deep integration with supraseal_c2: it calls supraseal_c2::generate_groth16_proof() in two places within the prover module, and it imports supraseal_c2::SRS for the structured reference string parameters. This is crucial evidence — it proves that bellperson already treats supraseal_c2 as an internal dependency and that adding re-export wrappers would be consistent with the existing architecture.

The grep results also reveal something subtler: the integration points are concentrated in bellperson/src/groth16/prover/supraseal.rs and bellperson/src/groth16/supraseal_params.rs. These are the natural locations to add alloc_gpu_mutex and destroy_gpu_mutex wrappers. The assistant's subsequent actions in [msg 2203] and [msg 2204] confirm this — it edits supraseal.rs to add the wrappers and then re-exports them from bellperson/src/groth16/mod.rs.

The Solution: Re-export as Architectural Pattern

The decision to re-export through bellperson rather than adding a direct dependency is a textbook example of respecting module boundaries in Rust. The reasoning is straightforward:

  1. Bellperson already depends on supraseal_c2, as confirmed by the grep. Adding wrapper functions adds no new dependency edges to the graph.
  2. The engine already imports from bellperson, specifically bellperson::groth16 types like SuprasealParameters and ProvingAssignment. Adding alloc_gpu_mutex and SendableGpuMutex to this import path is a natural extension.
  3. The pipeline module sits between the engine and bellperson, providing an additional layer of abstraction. The assistant's plan to "re-export through bellperson and then through the pipeline module" creates a clean chain: engine → pipeline → bellperson → supraseal-c2. This approach also has a practical benefit: the SendableGpuMutex wrapper type, which wraps a raw *mut c_void pointer and implements Send and Sync, can be defined once in bellperson and reused by both the pipeline module and the engine. Without this re-export, the engine would need to either define its own wrapper (duplicating code) or somehow access the bellperson type through a non-obvious path.

The Broader Significance

Message [msg 2202] is a pivot point in the Phase 8 implementation. Before this message, the assistant had been working linearly through a todo list, modifying files in dependency order (C++ → FFI → bellperson → pipeline). The engine changes were the last major piece, and this architectural constraint was the first obstacle in that final step.

The message also reveals something about the assistant's working style: when hitting a conceptual roadblock, it immediately reaches for a concrete verification tool — in this case, grep — to confirm its understanding of the dependency graph before proceeding. The grep output is not just a search result; it's evidence that validates the proposed solution path.

In the messages immediately following [msg 2202], the assistant proceeds to implement the re-export wrappers ([msg 2203] and [msg 2204]), update the engine to use the new bellperson re-exports ([msg 2205]), and then hit a series of Send-related compilation errors that required further debugging ([msg 2206] through [msg 2213]). The raw pointer inside SendableGpuMutex proved tricky to thread through tokio::spawn and spawn_blocking closures, ultimately requiring a usize cast workaround. But the architectural decision made in [msg 2202] — re-export through bellperson — held firm throughout those subsequent fixes.

Conclusion

Message [msg 2202] is a small but revealing moment in a complex engineering session. It demonstrates that effective implementation work is not just about writing code — it's about understanding the dependency architecture of the system and finding the cleanest path through it. The assistant's recognition that "the engine can't directly access supraseal_c2" is not a failure but an insight, one that leads to a more maintainable design where each layer of the abstraction stack respects its boundaries. The grep for supraseal_c2 is the tool that confirms the insight and lights the way forward, turning an architectural constraint into a clean, principled solution.