Chunk 27.1

In this chunk, the assistant implemented Phase 10 of the GPU optimization, introducing a two-lock design (`gpu_locks` struct with `mem_mtx` and `compute_mtx`) to better overlap CPU-side memory management with GPU kernel execution. The `mem_mtx` was intended to handle short VRAM allocation and pre-staging (~18ms), while `compute_mtx` serialized the actual GPU kernel runs. The code was restructured in `groth16_cuda.cu`, changing the FFI interface to pass the new opaque lock struct and setting `gpu_workers_per_device` to 3. Initial builds failed due to a type-casting error but were quickly fixed. However, correctness testing revealed severe OOM failures and performance regressions (prove time ballooning to 102s). Debugging the timing logs showed that only the first worker successfully pre-staged VRAM; subsequent workers found insufficient free memory because the first worker's 12 GiB allocation was still live. The root cause was that `cudaDeviceSynchronize` and `cudaMemPoolTrimTo` inside `mem_mtx` are device-global operations that blocked while another worker held `compute_mtx` and ran kernels. This effectively serialized the two locks, destroying the intended overlap and forcing all subsequent partitions to use the slow fallback path. The assistant ultimately diagnosed the fundamental conflict: memory management operations on a single CUDA device cannot be fully isolated from compute operations. The solution was to remove the device-wide synchronization from the `mem_mtx` region entirely, relying instead on the already-proven fallback path inside `compute_mtx` when pre-staging fails. This chunk highlights the critical theme of hardware constraints (VRAM contention, device-global synchronization) overriding software abstraction, teaching that naive lock splitting can introduce severe serialization if it relies on operations that implicitly synchronize the entire device.

When Hardware Overrides Abstraction: The Phase 10 Two-Lock Design and the Discovery of CUDA's Device-Global Trap 2129 words

Message Articles