The Six-Word Diagnostic That Reshaped a Pipeline

Message: Maybe we're not freeing b_g2_m mem?

Introduction

In the middle of a high-stakes optimization campaign targeting Filecoin's SUPRASEAL_C2 Groth16 proof generation pipeline, a single six-word question from the user — "Maybe we're not freeing b_g2_m mem?" — arrived at a moment of crisis. The team had just completed the Phase 12 split GPU proving API, a sophisticated architectural change that offloaded the b_g2_msm computation (a multi-scalar multiplication on the G2 curve) from the GPU worker's critical path onto a background thread. The initial benchmarks were promising: with 10 partition workers (pw=10), throughput improved from 38.0s to 37.1s per proof, a ~2.4% gain. But when the user asked to push synthesis parallelism higher — first to 15 partition workers, then to 12 — the system crashed with out-of-memory (OOM) errors, its 755 GiB of RAM proving insufficient. This message ([msg 2986]) represents a pivotal diagnostic intuition, one that would drive the next several hours of investigation and ultimately reshape the team's understanding of memory pressure in their pipeline.

Context: The Phase 12 Split API and the OOM Wall

To understand the weight of this message, one must appreciate the architecture it targeted. The Phase 12 split API ([msg 2910] onward) was designed to decouple the GPU worker's critical path from the CPU-bound b_g2_msm computation. In the prior Phase 11 design, each GPU worker would: (1) receive a synthesized partition, (2) run the GPU kernels for NTT and MSM, (3) wait for b_g2_msm to complete on the CPU, and (4) only then pick up the next partition. Phase 12 split this: the GPU worker would start the proof, launch b_g2_msm on a background thread, and immediately become available for the next partition. The b_g2_msm result would be collected later by a finalizer task. This promised better GPU utilization by reducing idle time.

The implementation involved changes across multiple layers: Rust FFI signatures, C++ CUDA code, the Go-bound engine worker loop, and the pipeline orchestration layer. The assistant fixed compilation errors — a missing SynthesisCapacityHint struct, a trait bound mismatch in prove_start, continue statements that needed to be return in async blocks — and discovered a critical use-after-free bug where a background thread captured a dangling reference to a stack-allocated array ([msg 2953]). After these fixes, the split API worked correctly at pw=10.

But the user's push for higher parallelism exposed a memory ceiling. At pw=15, the daemon OOM'd immediately. At pw=12, it OOM'd again after a few proofs. The assistant's initial diagnosis focused on raw arithmetic: each partition synthesis uses ~13 GiB, so 15 concurrent syntheses would need ~195 GiB just for synthesis, plus the 44 GiB SRS, 26 GiB PCE, and other overheads — totaling well over the 755 GiB available. This seemed like a straightforward capacity limit.

The User's Diagnostic Intuition

The user's message — "Maybe we're not freeing b_g2_m mem?" — cuts against that simple arithmetic explanation. It suggests something more subtle: not that there's too much memory required, but that memory is being held when it shouldn't be. The b_g2_msm computation involves large intermediate buffers. If the split API's background thread or the PendingProofHandle struct retains those buffers after they're no longer needed, the memory accumulates across partitions, creating a hidden multiplier.

This is a characteristically sharp systems intuition. The user is not asking "do we have enough RAM?" but rather "are we leaking or holding memory unnecessarily?" The question implicitly identifies a specific suspect — b_g2_m (the G2 MSM buffers) — and a specific mechanism — not freeing — that would explain why increasing partition workers from 10 to 12 (a modest 20% increase) causes catastrophic failure rather than graceful degradation.

The message also reveals an important assumption: that the Phase 12 split API, being new code, likely has a memory management bug. This is a reasonable heuristic — new code paths are the most common source of leaks. The user is betting that the bug is in the freshly written split logic rather than in the well-tested synthesis or GPU kernel code.

The Investigation That Followed

The assistant took this hypothesis seriously and launched a systematic investigation ([msg 2987] onward). The first step was to trace whether the PendingProofHandle (returned by prove_start) held memory unnecessarily after the function returned. By examining the C++ prep_msm_thread code, the assistant confirmed that the massive a, b, c NTT evaluation vectors — each ~12 GiB per partition — were only needed by the GPU kernel region, which completed before prove_start returned. An early deallocation of these vectors was implemented.

But the OOMs persisted at pw=12. This led to a deeper diagnostic effort: building a global buffer tracker with atomic counters (buf_synth_start, buf_abc_freed, buf_dealloc_done) integrated into the pipeline and engine, providing real-time visibility into every large buffer class in flight ([msg 2988]). This instrumentation revealed the true bottleneck: the provers counter peaked at 28, meaning 28 synthesized partitions were queued simultaneously, each holding their full ~16 GiB datasets.

The root cause was architectural, not a leak. The partition semaphore (set to pw=12) released immediately after synthesis completed, allowing new synthesis tasks to start. But the GPU channel had only a single slot, so synthesized partitions queued up waiting for GPU access. The semaphore permit was returned before the job was delivered to the GPU, creating a pile-up. The fix was to hold the semaphore permit until the synthesized job was fully delivered to the GPU channel, which dramatically reduced peak RSS from 668 GiB to 294.7 GiB and enabled pw=12 to run without OOM for the first time.

However, this fix introduced a throughput regression (39.9s vs 37.1s) by serializing synthesis and channel delivery. The assistant then reverted the semaphore change and instead increased the channel capacity from 1 to partition_workers, allowing a natural buffer of completed jobs without blocking the semaphore.

Input Knowledge Required

To understand this message, one needs: familiarity with the Groth16 proof generation pipeline, specifically the b_g2_msm computation and its role in the critical path; knowledge of the Phase 12 split API design and how it differs from Phase 11; awareness that the system has 755 GiB of RAM and that pw=10 works but pw=12 and pw=15 OOM; understanding of memory ownership patterns in C++/CUDA and how background threads can inadvertently retain buffers; and the mental model that memory pressure in a pipeline can come from either insufficient total capacity or from inefficient lifecycle management of individual allocations.

Output Knowledge Created

This message catalyzed a multi-phase investigation that produced: confirmation that the a, b, c NTT evaluation vectors could be freed early in prove_start (~12 GiB per partition reclaimed); a global buffer tracker with atomic counters providing real-time visibility into buffer classes; the discovery that the partition semaphore's early release was causing a queue pile-up of 28 synthesized partitions; and the insight that channel capacity (not just synthesis parallelism) is the critical control parameter for memory pressure. The final resolution — increasing GPU channel capacity from 1 to partition_workers — balanced memory and throughput, achieving 37.1s/proof without OOM.

The Deeper Lesson

The user's question was technically incorrect in its specific hypothesis — the OOM was not caused by b_g2_m memory being unfreed, but by a semaphore release timing issue. Yet it was profoundly correct in its general direction: the memory pressure was indeed a lifecycle management problem, not a raw capacity shortage. The investigation it triggered revealed that the system had enough RAM for pw=12 — it was just holding onto memory too long due to pipeline architecture. The six-word question embodied a debugging philosophy that separates the signal from the noise: when you hit a resource limit, ask not "do we have enough?" but "are we holding onto something we should have let go?"