Taming Memory in the GPU Proving Pipeline: Phase 12 Memory Backpressure

The Message That Closed the Loop

On February 20, 2026, the assistant posted a compact summary message ([msg 3243]) that marked the culmination of a focused engineering sprint to solve a critical memory explosion problem in the SUPRASEAL_C2 Groth16 proof generation pipeline. The message reads, in full:

All done. Committed at 98a52b33. Here's the summary:

>

## Phase 12 Memory Backpressure — Results

>

Three changes on top of the split API (99c31c2c):

>

1. Early a/b/c free — clears ~12 GiB/partition of evaluation vectors immediately after prove_start returns (GPU no longer needs them) 2. Channel capacity auto-scaling — sizes the synthesis→GPU channel to max(synthesis_lookahead, partition_workers) instead of hardcoded 1, preventing synthesis outputs from blocking on send while holding large allocations 3. Permit held through send — partition semaphore permit is held until after the channel send (not just through synthesis). With channel=pw, sends are non-blocking, so this adds zero latency but bounds total in-flight outputs

>

### Benchmark Results (20 proofs, j=20, gw=2, gt=32)

>

| pw | s/proof | Peak RSS | Notes | |----|---------|----------|-------| | 10 | 38.5 | 321 GiB | Down from 367 GiB | | 12 | 37.7 | 400 GiB | Was OOM at 668 GiB — fixed! | | 14 | 37.8 | 457 GiB | Similar throughput, more memory | | 16 | 38.4 | 510 GiB | Slower — DDR5 bandwidth wall |

>

Optimal config: pw=12 — best throughput-to-memory ratio. The memory fix reduced peak RSS by 40% (668→400 GiB) while enabling pw=12 to actually complete successfully.

At first glance, this reads as a straightforward status update: three changes were made, benchmarks were run, everything works. But beneath the surface, this message represents the resolution of a deep and subtle engineering problem — one that required understanding the interplay between asynchronous channel semantics, GPU memory ownership, semaphore-based admission control, and the peculiar memory footprint of Groth16 proof synthesis. This article unpacks the reasoning, assumptions, decisions, and knowledge embedded in this single message.

The Problem: Why Memory Was Exploding

To understand why this message was written, one must understand the crisis it resolved. The preceding message in the conversation ([msg 3212]) had shown that the Phase 12 split API — a major architectural change that decoupled GPU proof computation from CPU post-processing — was producing promising throughput (37.1s/proof in initial benchmarks) but suffered from catastrophic memory growth at higher parallelism levels. When partition_workers (pw) was set to 12, the daemon would consume 668 GiB of resident memory and crash with an out-of-memory (OOM) error. This was not a minor leak; it was a fundamental failure of the pipeline's admission control.

The root cause was a mismatch between the rate of synthesis (CPU-bound partition generation) and the rate of GPU consumption. The synthesis step produces partition outputs that each occupy roughly 12 GiB for the a/b/c evaluation vectors alone, plus additional data for density bitvecs and assignments. The original Phase 12 design used a semaphore to limit the number of concurrent synthesis tasks, but the channel connecting synthesis to GPU workers had a fixed capacity of 1. This created a dangerous dynamic: a synthesis task would acquire a semaphore permit, complete its work, and then block trying to send() the result into the full channel. While blocked on send(), the task still held its semaphore permit and retained ownership of all its memory allocations. The semaphore, intended to bound memory, became completely ineffective because permits were released before the channel send, not after. The result was unbounded memory accumulation as synthesis tasks piled up waiting for the GPU to drain the channel.

The Three Interventions: Reasoning and Design

The assistant's message describes three changes, each targeting a different layer of the memory problem. Understanding why each was necessary reveals the sophistication of the solution.

Early a/b/c free addressed the most obvious waste: the a/b/c evaluation vectors — each approximately 12 GiB per partition — were being retained long after the GPU had finished using them. The prove_start function hands off the partition data to the GPU for NTT (Number Theoretic Transform) and MSM (Multi-Scalar Multiplication) computation. Once those GPU kernels complete and cudaHostUnregister has run, the evaluation vectors serve no further purpose. Only the density bitvecs and assignment data are needed for the background b_g2_msm finalization. By clearing these vectors immediately after prove_start returns, the assistant reclaimed ~12 GiB per in-flight partition. This is a textbook example of "free early, free often" — a memory management principle that is easy to state but requires deep knowledge of the dataflow to apply correctly.

Channel capacity auto-scaling tackled the blocking-on-send problem. The original hardcoded channel capacity of 1 meant that only one synthesized partition could be buffered between synthesis and GPU consumption. When the GPU was busy, synthesis tasks would complete and immediately find the channel full. The fix sizes the channel to max(synthesis_lookahead, partition_workers), ensuring that all concurrently running synthesis tasks have room to deposit their results without blocking. This is a critical insight: the channel should be sized to the synthesis parallelism, not to some arbitrary small constant. The assumption that a small channel would naturally bound memory was incorrect because it ignored the fact that blocked send() calls still hold their payload in memory.

Permit held through send closed the final loophole. The semaphore permit — the mechanism intended to limit concurrent synthesis — was being released immediately after synthesis completed, before the channel send. This meant the permit counted a task as "done" while its output still occupied memory waiting to enter the channel. By holding the permit until after the send() succeeds, the assistant ensured that the number of in-flight synthesis outputs is strictly bounded by partition_workers. Since the channel now has capacity equal to partition_workers, the send() is non-blocking, so holding the permit adds zero latency. The semaphore and channel now work in concert: the semaphore limits how many synthesis tasks can run concurrently, and the channel buffers their outputs without allowing unbounded accumulation.

The Benchmarking Campaign: Empirical Validation

The message's benchmark table is the result of an extensive empirical campaign visible in the preceding conversation messages ([msg 3201] through [msg 3234]). The assistant systematically tested four values of partition_workers (10, 12, 14, 16) across multiple runs, measuring both throughput and peak RSS. Each configuration required restarting the daemon, waiting for SRS preloading, and running a 20-proof batch at concurrency 20.

The results tell a clear story. At pw=10, the system was stable at 38.5s/proof and 321 GiB — already improved from 367 GiB before the fix. At pw=12, the breakthrough: 37.7s/proof with 400 GiB peak RSS, down from an OOM crash at 668 GiB. This 40% memory reduction was the primary victory. At pw=14, throughput was essentially identical (37.8s/proof) but memory climbed to 457 GiB. At pw=16, throughput actually regressed to 38.4s/proof while memory hit 510 GiB. The assistant correctly diagnosed the regression as hitting the "DDR5 bandwidth wall" — more parallelism creates more memory contention on the CPU side, slowing down synthesis and offsetting any GPU-side gains.

The optimal configuration, pw=12, represents the point where synthesis parallelism is high enough to keep the GPU fed without overwhelming the memory subsystem. This is not a universal constant; it is specific to this machine's DDR5 bandwidth, GPU count (2), GPU threads (32), and the 32 GiB sector size of the Filecoin PoRep circuit. The assistant's methodology — sweeping pw values and measuring both throughput and memory — is the correct way to find this optimum.

Assumptions and Their Validity

The message and its surrounding context reveal several assumptions, some explicit and some implicit.

The assistant assumed that the DDR5 bandwidth wall, not GPU compute capacity, would be the limiting factor at high pw values. This was validated by the pw=16 regression: more synthesis workers created more memory traffic, slowing each worker down without improving GPU utilization. The assumption that channel capacity should equal partition_workers was based on the insight that all concurrent synthesis tasks should be able to enqueue their results simultaneously. This holds as long as the channel consumer (GPU worker) processes items at least as fast as they are produced, which the benchmarks confirmed.

An implicit assumption is that cudaHostUnregister truly completes before prove_start returns, making the a/b/c vectors safe to free. This depends on correct CUDA stream synchronization in the underlying C++ code. If the GPU kernels were still reading from the host-mapped buffers when the vectors were cleared, the results would be corrupted. The assistant's confidence in this assumption is based on the architecture of the split API, where prove_start is designed to be synchronous from the caller's perspective — it blocks until GPU work is done.

Another assumption is that the benchmark methodology (20 proofs, concurrency 20, single warm-up run) produces representative results. The assistant ran multiple trials (pw=12 was tested twice, yielding 37.7s and 38.5s), suggesting awareness that individual runs have variance. The 37.7s result from the first pw=12 run may have benefited from favorable memory layout; the second run at 38.5s is more conservative.

Knowledge Required and Knowledge Created

To fully understand this message, a reader needs knowledge of several domains: asynchronous channel semantics in Rust (the crossbeam or tokio channels used for synthesis→GPU communication), semaphore-based admission control patterns, the Groth16 proving pipeline structure (synthesis produces partitions, GPU consumes them for NTT/MSM, CPU finalizes with b_g2_msm), CUDA memory management (cudaHostRegister for zero-copy access), and the memory profile of Filecoin PoRep proofs (~12 GiB per partition for evaluation vectors).

The message creates new knowledge in the form of a validated memory backpressure design pattern. The key insight — that a semaphore permit must be held through the channel send, not just through computation, and that channel capacity must match the permit count — is transferable to any producer-consumer pipeline where the producer holds significant memory. The benchmark results also create a concrete data point: on this hardware, pw=12 with gw=2 and gt=32 is optimal for 32 GiB sectors, achieving 37.7s/proof at 400 GiB RSS.

The Thinking Process

The reasoning visible in the surrounding messages shows a methodical, hypothesis-driven approach. The assistant first identified the OOM at pw=12 as the critical problem ([msg 3212]), then diagnosed the mechanism (synthesis outpacing GPU, permits released before send). The three fixes were designed to address specific failure modes: early free for the a/b/c retention, channel scaling for the blocking-on-send, and permit-hold for the semaphore loophole. Each fix was implemented in code ([msg 3238] shows the diff), then validated through the benchmarking sweep. The assistant did not stop at pw=12 working; it explored pw=14 and pw=16 to find the true optimum and understand the limiting factor (DDR5 bandwidth). The commit message ([msg 3240]) is unusually detailed, documenting not just what was changed but why, and including the full benchmark table. This level of documentation reflects an engineering culture where the commit log serves as a design record.

Conclusion

Message [msg 3243] is a summary, but it is also a closing argument. It demonstrates that a seemingly intractable memory explosion — 668 GiB and OOM — was resolved through careful analysis of asynchronous control flow, not through brute-force memory reduction. The three changes are individually simple but collectively powerful: early free reclaims memory, channel scaling prevents backpressure deadlocks, and permit-hold closes the admission control loophole. The result is a pipeline that runs faster (37.7s vs. the previous best) with less memory (400 GiB vs. OOM at 668 GiB). This is the kind of engineering that looks inevitable in retrospect but required deep understanding of the system's dataflow, memory ownership model, and concurrency architecture to achieve.