The Raw Pointer That Wouldn't Cross the Async Boundary: A Rust Send-Safety Debugging Epiphany

Introduction

In the middle of implementing Phase 8: Dual-Worker GPU Interlock for the cuzk SNARK proving engine, the assistant hit a wall. A Rust compilation error, cryptic and stubborn, refused to let a carefully designed mutex-sharing scheme cross the boundary between synchronous computation and asynchronous task spawning. Message [msg 2210] captures the moment of diagnosis—a short but surgically precise insight that untangled the knot. This article examines that message in depth: why it was written, what assumptions it challenged, and what it reveals about the friction between Rust's safety guarantees and the raw-pointer world of CUDA/FFI.

The Scene: Phase 8 Dual-Worker GPU Interlock

By message [msg 2210], the assistant had already completed the bulk of a substantial architectural change spanning seven files and roughly 195 lines of code. Phase 8 aimed to eliminate GPU idle gaps in the Groth16 proof generation pipeline by narrowing a C++ static mutex to cover only the CUDA kernel region (NTT+MSM, batch additions, tail MSMs). CPU preprocessing and the b_g2_msm computation would now run outside the lock, allowing two GPU workers per device to interleave their execution—one performing CPU work while the other occupied the GPU.

The implementation required threading a C++ std::mutex pointer through a multi-layer FFI chain: from the CUDA kernel code in groth16_cuda.cu, through the supraseal-c2 Rust FFI wrapper, into the bellperson Groth16 prover, and finally into the engine's worker spawn logic in pipeline.rs. A SendableGpuMutex wrapper type was created to make the raw *mut c_void pointer (representing the C++ mutex) compatible with Rust's concurrency traits. Helper functions alloc_gpu_mutex and destroy_gpu_mutex were added to the CUDA code to allocate and free these mutexes from the Rust side.

The engine was modified to create one C++ mutex per GPU, then spawn gpu_workers_per_device workers (default 2) per GPU, each sharing the same mutex pointer. This is where the trouble began.

The Compilation Error That Wouldn't Budge

When the assistant attempted to build the modified code ([msg 2206]), the Rust compiler rejected the change:

error: *mut c_void` cannot be sent between threads safely
  --> cuzk-core/src/engine.rs:1354:61

The spawn_blocking closure required the Send trait, but the raw pointer *mut c_void extracted from SendableGpuMutex did not implement it. The assistant's initial diagnosis ([msg 2207]) correctly identified the issue: SendableGpuMutex was marked Send, but extracting .0 to obtain the raw pointer inside the closure lost that guarantee. The fix attempted in [msg 2208]—moving the entire SendableGpuMutex into the closure—failed because the closure still accessed .0 internally, exposing the raw pointer to the compiler's trait checker.

A second rebuild ([msg 2209]) produced the same error, now pointing at the same line with more precise context. The compiler was unyielding.

Message 2210: The Diagnosis

It is at this point that the assistant writes message [msg 2210]:

The SendableGpuMutex contains a *mut c_void internally, which prevents the closure from being Send. Even though SendableGpuMutex is marked Send, the closure captures it and then accesses .0 which is the raw pointer. The issue is the raw pointer is directly in the closure's captures.

>

The fix: convert the pointer to a usize (which is Send) and cast back: [edit] /home/theuser/curio/extern/cuzk/cuzk-core/src/engine.rs Edit applied successfully.

This message is deceptively short—only three sentences of reasoning before the edit command. But within those sentences lies a precise understanding of Rust's trait system, the interaction between Send and closure capture semantics, and a pragmatic workaround that respects the compiler's constraints while achieving the desired runtime behavior.

Why This Message Was Written: The Reasoning Process

The assistant's reasoning unfolds in three layers.

Layer 1: Identifying the root cause. The assistant had previously assumed that moving the entire SendableGpuMutex into the closure would solve the problem, since the wrapper type implements Send. But the compiler rejected that too. Message [msg 2210] refines the diagnosis: the issue is not merely that a raw pointer is used inside the closure, but that the raw pointer is directly present in the closure's captures. When Rust compiles a closure, it generates an anonymous struct containing copies of all captured variables. If SendableGpuMutex is captured, the closure's capture struct contains a *mut c_void field. Even though SendableGpuMutex implements Send (via an explicit unsafe impl), the compiler sees the raw pointer field in the capture struct and rejects it because *mut c_void does not implement Send.

This is a subtle point. The Send implementation on SendableGpuMutex is an unsafe assertion by the programmer that the type can be transferred between threads safely. But when the type is destructured—when its fields become part of another struct (the closure's capture struct)—that unsafe assertion does not automatically propagate to the field types. The compiler reverts to checking the field types directly.

Layer 2: Choosing the fix. The fix—converting the pointer to usize before capture and casting back inside the closure—is elegant. usize implements Send because it is a plain integer with no ownership semantics. The conversion is safe because the pointer is only used to pass the mutex address to the C++ code; no Rust-side dereferencing occurs. The cast back to *mut c_void inside the closure is an unsafe operation, but it is confined to a small, auditable scope.

Layer 3: Anticipating the next error. Interestingly, the assistant's fix in [msg 2210] was not the final solution. A subsequent build ([msg 2211]) revealed that the async move block itself (not just the inner spawn_blocking closure) captured gpu_mutex and failed the Send check. The assistant refined the approach in [msg 2212]: capture the mutex address as a usize before entering the async block, so the async block's capture struct contains only the integer, not the wrapper type with its raw pointer field.

Assumptions Made and Lessons Learned

The assistant made several assumptions during this debugging sequence:

  1. That SendableGpuMutex's Send impl would propagate through closure capture. This assumption was incorrect. Rust's trait system does not automatically decompose wrapper types when checking closure captures; it checks the capture struct's fields directly.
  2. That the spawn_blocking closure was the only problematic capture. This was partially correct—the raw pointer issue manifested at the spawn_blocking boundary first—but the outer async move block had the same problem. The assistant discovered this through iterative compilation.
  3. That a usize cast is a valid workaround. This assumption was correct. The C++ mutex pointer is never dereferenced from Rust; it is passed opaquely through FFI. The integer representation preserves all necessary information and satisfies the Send requirement. The deeper lesson is about the tension between Rust's safety guarantees and the realities of FFI. When interfacing with C++ code, raw pointers are unavoidable. Rust's trait system provides escape hatches (unsafe impl Send), but these hatches have sharp edges. A type can be Send while its fields are not—this is legal because the unsafe impl is a promise that the type as a whole can be safely transferred. However, when the type is destructured (as happens in closure capture), the compiler forgets this promise and checks the fields individually. This is not a compiler bug; it is a deliberate design choice that prevents unsafe code from accidentally circumventing trait bounds through wrapper types.

Input and Output Knowledge

To understand message [msg 2210], the reader needs:

The Broader Significance

Message [msg 2210] is a microcosm of a larger theme in the cuzk proving engine project: the constant negotiation between performance and safety. The dual-worker GPU interlock is a performance optimization—it eliminates GPU idle time by allowing two workers to share a GPU, interleaving CPU preprocessing with CUDA kernel execution. But implementing this optimization requires threading a raw C++ mutex pointer through Rust's async runtime, which demands careful attention to Rust's safety invariants.

The usize cast workaround is pragmatic but not beautiful. It works because the pointer is never dereferenced on the Rust side, but it introduces an unsafe cast that must be carefully maintained. A more elegant solution would involve restructuring the code to avoid capturing the pointer altogether—perhaps by passing it through a channel or storing it in a global registry. But in the context of a high-performance proving engine where every microsecond counts, the usize cast is the right tradeoff: minimal overhead, minimal code change, and a clear safety justification.

Conclusion

Message [msg 2210] captures a moment of clarity in a frustrating debugging session. The assistant correctly identifies that Rust's closure capture semantics interact poorly with unsafe impl Send on wrapper types containing raw pointers, and devises a minimal fix that respects the compiler's constraints. The message is brief—only three sentences of reasoning—but those sentences encode a deep understanding of Rust's type system, async concurrency, and the practical realities of FFI programming. It is a reminder that even in a project dominated by CUDA kernels and memory optimization, the most stubborn bugs can live in the type checker.