The Turning Point: Implementing Phase 10's Two-Lock Design

In the long-running optimization campaign for Filecoin's PoRep C2 proof generation pipeline, message [msg 2589] marks a critical inflection point—the moment when careful design reasoning meets the unforgiving reality of CUDA hardware constraints. This message, brief in its surface form but loaded with consequence, represents the assistant's first implementation step for Phase 10: a two-lock architecture intended to solve a newly discovered CPU memory bandwidth bottleneck. What makes this message particularly compelling is not what it contains, but what it sets in motion—an implementation that will soon reveal a fundamental conflict between software abstraction and GPU device semantics, leading to OOM failures, performance regressions, and a deeper understanding of what it means to synchronize operations on a single CUDA device.

The Context: A Bottleneck Shift

To understand why this message was written, one must trace the optimization journey that preceded it. The project had been systematically working through a series of bottlenecks in the Groth16 proof generation pipeline for Filecoin's Proof-of-Replication (PoRep). Phase 9 had successfully optimized PCIe transfers, achieving a 14.2% throughput improvement in single-worker mode. But when the team ran extensive benchmarks with higher concurrency (c=15–30), a new bottleneck emerged: the CPU critical path—specifically prep_msm (1.9s) and b_g2_msm (0.48s)—now dominated the per-partition wall time at ~2.4s, leaving the GPU idle for approximately 600ms per partition waiting for the CPU thread. At high concurrency, the ten synthesis workers competed with CPU MSM operations for 8-channel DDR5 memory bandwidth, inflating CPU times by 2–12×.

The bottleneck had shifted from PCIe transfers and GPU kernel execution to CPU memory bandwidth contention. This was a fundamentally different kind of problem—one that couldn't be solved by optimizing GPU kernel code or transfer patterns alone. The solution required architectural changes to how CPU and GPU work overlapped.

The Design: Two Locks Instead of One

The assistant's response to this bottleneck shift was to design Phase 10, documented in c2-optimization-proposal-10.md ([msg 2587]). The core insight was elegant: split the existing single std::mutex into a two-lock structure with separate mutexes for memory management (mem_mtx) and GPU compute (compute_mtx). The reasoning was that VRAM allocation and pre-staging uploads were fast (~18ms), while GPU kernel execution was the dominant time consumer (~1.8s). If these two phases could overlap across different workers, the CPU overhead could be hidden behind GPU compute.

The design was carefully analyzed for deadlock risks in [msg 2583]. The assistant identified a potential resource deadlock on VRAM—if one worker held compute_mtx with 12 GiB of live allocations while another worker held mem_mtx trying to allocate the same 12 GiB, the second worker would OOM. But the analysis revealed a simplifying insight: because d_a and d_bc (the two large device buffers totaling ~12 GiB) were freed synchronously via cudaFree inside compute_mtx, the VRAM was genuinely available by the time compute_mtx was released. No cooperative free slot was needed. The flow per worker became clean:

  1. No lock: cudaHostRegister + prep_msm_thread starts
  2. mem_mtx: Pool trim, memory info check, cudaMalloc 12 GiB, async upload, events
  3. compute_mtx: GPU kernels (NTT, batch_add, tail_msm), synchronous cudaFree of d_a and d_bc
  4. No lock: Destroy events/stream, cudaHostUnregister, join prep_msm_thread This design was committed to the proposal document, and the assistant updated its todo list to begin implementation.

The Implementation Begins

Message [msg 2589] is the first concrete implementation step. Its full text reads:

[assistant] Now let me implement. First, the gpu_locks struct and updated FFI functions: [edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu Edit applied successfully.

The message contains a single tool call—an edit to groth16_cuda.cu. This edit defines the gpu_locks struct (containing compute_mtx and mem_mtx) and updates the create_gpu_mutex/destroy_gpu_mutex FFI functions to allocate and free this new struct instead of the old single std::mutex. The Rust FFI boundary remains unchanged—the opaque pointer type *mut c_void is preserved, so no changes are needed in lib.rs, bellperson, or cuzk-core.

This is a deceptively small change for such a significant architectural shift. The struct definition is the foundation upon which the entire Phase 10 overlap strategy rests. Without it, the restructured lock regions in generate_groth16_proofs_c cannot exist.

Assumptions Embedded in the Implementation

The implementation carries several assumptions, some explicit and some implicit:

Assumption 1: Lock separation is feasible on a single CUDA device. The core premise of Phase 10 is that memory management operations (allocation, upload, pool trim) can be cleanly separated from compute operations (kernel launches, synchronization) using two different mutexes. This assumes that CUDA's device-wide state machine allows these operations to proceed independently—that one worker can allocate memory while another runs kernels on the same device.

Assumption 2: cudaMemPoolTrimTo without cudaDeviceSynchronize is sufficient. The design deliberately removed cudaDeviceSynchronize from the mem_mtx region, reasoning that pool trim alone would reclaim stream-ordered pool memory because all stream operations were synchronized inside compute_mtx before the mutex was released. This assumption would prove incorrect.

Assumption 3: Three workers per GPU is the right concurrency level. The design increased gpu_workers_per_device from 2 to 3, based on the expected overlap: with three workers, one could be in mem_mtx (18ms), one in compute_mtx (1.8s), and one in CPU preprocessing (1.9s), all running concurrently. This assumed that three-way overlap was achievable without resource contention.

Assumption 4: The fallback path is a safe escape. The design included a fallback: if cudaMalloc fails in the mem_mtx region (not enough VRAM), the worker would skip pre-staging and allocate inside compute_mtx using the original non-prestaged path. This fallback was assumed to be a safe, albeit slower, alternative.

The Knowledge Required

To understand this message, one needs knowledge spanning several domains:

The Knowledge Created

This message creates the first concrete artifact of Phase 10: the gpu_locks struct and updated FFI functions in groth16_cuda.cu. This is the foundation that subsequent edits ([msg 2590] through [msg 2597]) will build upon to restructure the entire generate_groth16_proofs_c function.

But more importantly, this message creates the expectation that the two-lock design will work. The assistant proceeds with confidence, updating its todo list to mark the struct implementation as complete and moving on to the main function restructure. The build succeeds ([msg 2608]), the daemon starts with three workers ([msg 2610]), and everything looks promising.

The Irony: What Follows

The tragedy of this message is that the implementation it inaugurates will fail in a spectacular and instructive way. In [msg 2611], the correctness test fails with a 41.1s timeout. The daemon log reveals the root cause: cudaDeviceSynchronize inside mem_mtx blocks while another worker holds compute_mtx and runs kernels, effectively serializing the two locks and destroying the intended overlap. Subsequent workers find insufficient free VRAM because the first worker's 12 GiB allocation is still live. The fallback path then OOMs because the pool hasn't been properly trimmed.

The assistant's debugging journey ([msg 2613] through [msg 2627]) reveals the fundamental conflict: memory management operations on a single CUDA device cannot be fully isolated from compute operations. Operations like cudaDeviceSynchronize and cudaMemPoolTrimTo are device-global—they implicitly synchronize all streams, all kernels, and all memory operations on the device. No amount of mutex splitting can separate what the hardware treats as a single shared resource.

The eventual solution is to remove device-wide synchronization from the mem_mtx region entirely, relying on the already-proven fallback path inside compute_mtx when pre-staging fails. This is a humbling outcome: after all the careful design analysis, the implementation reveals that the hardware's semantics override the software abstraction.

Conclusion

Message [msg 2589] is the moment when Phase 10 transitions from design to implementation. It represents the assistant's confidence in a carefully reasoned solution to a real bottleneck. But it also represents a fundamental tension in GPU programming: the tension between the abstractions we build (mutexes, lock regions, concurrent workers) and the reality of the hardware (a single device with global state, implicit synchronization, and shared resources). The message is brief, but it sets in motion a chain of events that will teach a lasting lesson about the limits of software isolation on CUDA devices.