The Moment Theory Meets Hardware: Implementing Phase 10's Two-Lock Architecture

In the long arc of optimizing the SUPRASEAL_C2 Groth16 proof generation pipeline for Filecoin's Proof-of-Replication (PoRep), few messages capture the tension between elegant software design and unforgiving hardware constraints as vividly as message 2590. The message itself is deceptively brief — a single edit command with a short explanatory comment:

Now restructure the main function. The key change: replace the single gpu_lock with mem_lock (around pre-staging) and compute_lock (around GPU kernels). Let me update the fallback mutex too: [edit] /home/theuser/curio/extern/supraseal-c2/cuda/groth16_cuda.cu Edit applied successfully.

This is the implementation step for Phase 10 of the GPU optimization campaign. It looks like a routine code edit, but it represents the culmination of extensive reasoning, the testing of a carefully crafted hypothesis, and — as the subsequent messages reveal — the moment when a fundamental assumption about CUDA's programming model collides with reality.

The Bottleneck That Drove the Design

To understand why this message was written, one must understand the state of the optimization effort at this point. The project had progressed through nine phases of increasingly sophisticated optimization. Phase 9 had achieved a significant milestone: PCIe transfer optimization that reduced GPU-side kernel time to approximately 1.8 seconds per partition. However, the overall throughput plateaued at roughly 41 seconds per proof, and detailed timing instrumentation revealed a troubling bottleneck shift.

The critical path was no longer on the GPU. Instead, CPU-side operations — specifically prep_msm (1.9 seconds) and b_g2_msm (0.48 seconds) — now dominated the per-partition wall time at approximately 2.4 seconds. The GPU sat idle for roughly 600 milliseconds per partition, waiting for the CPU thread to finish its work. At high concurrency, the ten synthesis workers competed with the CPU MSM operations for the system's 8-channel DDR5 memory bandwidth, inflating CPU times by 2–12×. The bottleneck had moved from PCIe transfers and GPU kernel execution to CPU memory bandwidth contention.

The insight that drove Phase 10 was that the CPU-side work and GPU-side work could be overlapped across multiple workers. If worker A could be running GPU kernels while worker B was doing its CPU-side memory management (pre-staging VRAM allocations and uploads), the per-partition wall time could be reduced from ~3.7 seconds to an estimated ~1.8–2.0 seconds — a potential 30–38% throughput improvement.

The Design That Preceded the Code

Message 2590 did not emerge from thin air. It was the product of an extensive design process spanning multiple earlier messages. The assistant had:

Analyzed the timing data from Phase 9 benchmarks, identifying the CPU memory bandwidth bottleneck.
Designed a two-lock protocol to replace the single std::mutex that had previously serialized all GPU access. The new design used a gpu_locks struct containing two mutexes: mem_mtx (for VRAM allocation and pre-staging upload, estimated ~18ms) and compute_mtx (for GPU kernel execution, ~1.8s).
Analyzed deadlock risks in detail, concluding that since the large VRAM buffers (d_a and d_bc, totaling 12 GiB) were freed synchronously via cudaFree inside the compute_mtx region, no cooperative free slot mechanism was needed. The next worker could safely allocate in mem_mtx without waiting for the previous worker's compute to finish.
Written the design document c2-optimization-proposal-10.md, committing the architecture to the repository.
Implemented the gpu_locks struct and updated the FFI functions in the preceding message (msg 2589). Message 2590 was the next logical step: restructure the main function generate_groth16_proofs_c to actually use the two locks, replacing the single gpu_lock acquisition with separate mem_lock and compute_lock scopes.

The Assumptions Embedded in the Code

The edit in message 2590 encoded several critical assumptions about how CUDA devices behave:

Assumption 1: Memory management is independent of compute. The design assumed that calling cudaMalloc, cudaMemPoolTrimTo, and cudaDeviceSynchronize inside the mem_mtx region would only affect the current worker's memory operations and would not interfere with another worker's kernel execution happening under compute_mtx. This seemed reasonable — CUDA exposes streams as a mechanism for concurrent operations, and the API documentation suggests that memory management functions operate on the device's memory pool, not on specific streams.

Assumption 2: Pool trim without device synchronization is sufficient. The design removed cudaDeviceSynchronize from the mem_mtx region (based on the reasoning that by the time compute_mtx was released, all stream operations had been synchronized through implicit syncs in the MSM operations). The plan was to rely solely on cudaMemPoolTrimTo to reclaim cached memory from stream-ordered frees.

Assumption 3: The fallback path provides safety. The code included a fallback path: if pre-staging failed (due to insufficient VRAM), the worker would fall back to the original non-prestaged allocation path inside compute_mtx. This had been proven to work correctly in Phase 8.

Assumption 4: Three workers can productively share the GPU. The design increased gpu_workers_per_device from 2 to 3, expecting that the overlap of CPU and GPU work would hide the CPU overhead.

The Hardware Reality That Shattered the Assumptions

The edit in message 2590 compiled successfully, and the assistant proceeded to build and test. What followed was a cascade of failures that revealed the true nature of CUDA's device model.

The correctness test with c=1 j=1 (a single proof) failed immediately with an OOM error. The daemon log showed a pattern that told the whole story:

Worker 0 acquired mem_mtx, allocated 12 GiB successfully (prestage OK), and released mem_mtx.
Worker 1 acquired mem_mtx, but cudaMemGetInfo reported only ~1.5 GiB free — Worker 0's 12 GiB allocation was still live on the device. Pre-staging was skipped.
Worker 2 encountered the same situation.
Worker 0 then acquired compute_mtx and ran its kernels.
When Worker 1 finally acquired compute_mtx, the fallback path tried to allocate 8 GiB for d_b via synchronous cudaMalloc — and OOM'd. The root cause was that cudaDeviceSynchronize and cudaMemPoolTrimTo inside mem_mtx are device-global operations. When Worker 1 held mem_mtx and called cudaDeviceSynchronize, it blocked until ALL GPU operations on the device completed — including Worker 0's kernels running under compute_mtx. This effectively serialized the two locks, destroying the intended overlap entirely. The mem_mtx was not a lightweight 18ms hold; it became a blocking wait for the entire duration of the other worker's compute. The assistant attempted a fix by adding cudaDeviceSynchronize back into mem_mtx, but this only made the serialization explicit. The proof then passed correctness but took 102 seconds — a massive regression from the Phase 9 baseline of ~41 seconds. Timing logs showed that only the first partition pre-staged successfully; all subsequent partitions fell back to the slow non-prestaged path with gpu_total_ms of 3.3–3.9 seconds (Phase 8 performance).

The Deeper Lesson: Device-Global Synchronization

The debugging that followed message 2590 led to a fundamental realization: memory management operations on a single CUDA device cannot be fully isolated from compute operations. The functions cudaDeviceSynchronize and cudaMemPoolTrimTo operate at the device level, not the stream level. They interact with all streams, all contexts, and all pending operations on the device. No amount of lock splitting at the application level can separate what the hardware treats as a single shared resource.

The solution, as the assistant ultimately discovered, was to remove the device-wide synchronization from the mem_mtx region entirely. Instead of trying to guarantee that VRAM was available before entering compute_mtx, the code would simply attempt the allocation in mem_mtx without synchronization, and if it failed, rely on the already-proven fallback path inside compute_mtx. This meant accepting that the two-lock design could not achieve perfect overlap — the hardware constraints imposed a fundamental serialization point.

Input and Output Knowledge

To understand message 2590, one needs knowledge of several domains: the CUDA programming model (streams, device synchronization, memory pools), the existing codebase structure (the FFI boundary between Rust and C++, the groth16_cuda.cu file's organization, the gpu_ptr_t and dev_ptr_t RAII wrappers), the performance characteristics of the Groth16 proof pipeline (the ~18ms pre-staging time, the ~1.8s GPU kernel time, the 12 GiB VRAM requirement), and the lock protocol design that preceded the implementation.

The message itself produced a restructured generate_groth16_proofs_c function with two lock regions. But more importantly, it produced negative knowledge — the understanding that the two-lock design as conceived could not work on a single CUDA device. This negative knowledge was arguably more valuable than a successful implementation would have been, because it revealed a fundamental constraint of the hardware that would inform all subsequent optimization attempts.

The Thinking Process

While message 2590 itself contains minimal reasoning (just the brief comment about replacing gpu_lock with mem_lock and compute_lock), the thinking process is visible in the surrounding messages. The assistant had worked through the lock protocol design in detail, considering deadlock scenarios, analyzing the lifetime of VRAM allocations, and verifying that d_a and d_bc were freed synchronously inside compute_mtx. The reasoning was methodical and thorough — yet it failed to anticipate the device-global nature of CUDA synchronization primitives.

This is a classic case of the "leaky abstraction" problem. The CUDA API presents memory management functions as if they operate on independent resources, but underneath, they interact with a shared device state. The assistant's reasoning was correct at the level of the C++ memory model (mutexes, lock ordering, resource lifetimes) but incomplete at the level of the hardware (device-global synchronization, shared VRAM, implicit barriers). The message marks the boundary where software abstraction meets hardware reality — and reality wins.

Conclusion

Message 2590 is a pivotal moment in the optimization campaign. It is the point where a well-reasoned, carefully designed optimization strategy is committed to code and tested against real hardware. The edit itself is simple — a few lines of C++ restructuring — but it carries the weight of extensive analysis and the risk of untested assumptions. The message's true significance lies not in what it changed in the code, but in what the subsequent testing revealed about the fundamental nature of GPU programming: that some abstractions cannot be cleanly separated, and that the hardware's constraints will always override the software's intentions.