The Memory Ceiling: When Parallelism Meets Physics in GPU Proving

In the high-stakes world of Filecoin proof generation, every millisecond counts. The SUPRASEAL_C2 pipeline, a Groth16 proving system for Filecoin's Proof-of-Replication (PoRep), had just received a significant architectural upgrade: Phase 12's split GPU proving API, which offloaded the b_g2_msm computation from the GPU worker's critical path. Early benchmarks were promising — 37.1 seconds per proof with 10 partition workers. But the question lingered: could more parallelism squeeze out even better performance? The answer arrived in the form of a single, deceptively simple bash command that would reveal a hard physical constraint.

The Message

The subject message ([msg 2983]) is a benchmark execution, issued by the AI assistant after the user suggested trying higher synthesis parallelism:

/home/theuser/curio/extern/cuzk/target/release/cuzk-bench batch \
  --type porep --c1 /data/32gbench/c1.json \
  --count 20 --concurrency 15 2>&1

The output begins promisingly:

=== Batch Benchmark ===
proof type:  porep
count:       20
concurrency: 15

  [1/20] COMPLETED — 78.3s (prove=71880 ms, queue=305 ms)
  [2/20] COMPLETED — 119.2s (prove=65763 ms, queue=767 ms)
  [3/20] COMPLETED — 158.8s (prove=69734 ms, queue=1234 ms)
  [4/20] COMPLETED — 192.2s (prove=73763 ms, queue=1680 ms)
  [5/20] COMPLETED — 210.8s (prove=6...

The output is truncated — cut off mid-line at the fifth completion — because the daemon crashed. The system had run out of memory.

The Context: A Delicate Balance

To understand why this message matters, we need to trace the decisions that led to it. The assistant had just completed implementing Phase 12, a split GPU proving API that decoupled the b_g2_msm (a multi-scalar multiplication on the G2 curve) from the GPU worker's synchronous critical path ([msg 2961]). The initial benchmark with partition_workers=10 (pw=10) yielded 37.1 seconds per proof, a ~2.4% improvement over the Phase 11 baseline of 38.0 seconds.

The user then asked a natural question: "Try higher synthesis partition_workers in config, maybe 15/20?" ([msg 2964]). The reasoning was sound — if the GPU worker was now finishing faster thanks to the split API, the bottleneck might shift to synthesis throughput. More concurrent partition syntheses could keep the GPU fed with work, reducing idle time and improving overall throughput.

The assistant responded by systematically testing higher values. First, pw=15 was tried ([msg 2975]), which immediately failed with a connection error — the daemon had been killed by the OOM killer. The user confirmed: "oom" ([msg 2977]). The assistant then compromised on pw=12 as a middle ground ([msg 2979]), reasoning that 12 concurrent syntheses at ~13 GiB each would consume ~156 GiB, which combined with the SRS (44 GiB), PCE (26 GiB), and other overhead might still fit within the 755 GiB system.

What the Output Reveals

The truncated output tells a story more damning than a clean crash. The first five proofs show prove times of 71.9, 65.8, 69.7, 73.8, and approximately 67 seconds (truncated). Compare this to the 37.1 seconds achieved with pw=10. The system is already thrashing — memory pressure is so severe that the proving pipeline is spending significant time in kernel memory management, swapping, and garbage collection. The queue times are also elevated (305–1680 ms), suggesting contention in the job dispatch system.

The prove times are nearly double the pw=10 baseline. This is a textbook symptom of memory bandwidth contention: when the system exceeds its comfortable working set, the operating system's memory management overhead dominates, and every allocation becomes a gamble. The 755 GiB machine, which seemed generously provisioned, was revealed to be at its absolute limit.

The Assumptions Under Test

This message represents a critical assumption being tested and falsified. The assumption was: the split API has removed the GPU bottleneck, so increasing synthesis parallelism will improve throughput. This assumption rested on two premises:

1. The GPU worker was the bottleneck before Phase 12. This was well-supported by prior analysis ([msg 2961]), which showed the GPU worker's critical path included the b_g2_msm computation.
2. Synthesis is CPU-bound and can scale with more workers. This was partially true — synthesis is CPU-intensive — but it ignored the memory cost of each concurrent synthesis. The falsification was brutal: not only did pw=12 fail to improve throughput, it regressed performance by nearly 2× while also risking system stability. The assumption that "more parallelism = more throughput" collided with the physical reality of finite RAM.

A Mistake in Framing

There was a subtle mistake in how the problem was framed. The assistant and user were thinking in terms of compute bottlenecks — CPU synthesis throughput vs. GPU proving throughput. But the actual bottleneck at pw=12 was memory capacity. Each partition synthesis holds ~13 GiB of working data (the a, b, c NTT evaluation vectors, plus proving assignments and circuit values). With 12 concurrent syntheses, that's ~156 GiB. Add the SRS preload (44 GiB), PCE data (26 GiB), 15 concurrent job parsing contexts (~3 GiB each = ~45 GiB), and the GPU's working buffers, and the system's 755 GiB starts looking tight.

The mistake was not in trying higher parallelism — that's a standard optimization technique. The mistake was in not accounting for the memory ceiling before running the experiment. A quick calculation of the peak memory footprint would have predicted the OOM. However, the exact memory profile of each component was still being characterized, and the Phase 12 code had just been written. Some empirical exploration was unavoidable.

The Thinking Process Visible

The assistant's reasoning is visible across the sequence of messages leading to this point. After the pw=15 OOM ([msg 2975]), the assistant immediately calculated: "Each partition synthesis uses ~13 GiB, so 15 × 13 = ~195 GiB just for synthesis, plus the SRS (44 GiB) + PCE (26 GiB) + other overhead exceeds the 755 GiB" ([msg 2979]). This shows a clear mental model of the memory budget.

The decision to try pw=12 was a pragmatic compromise: "Let me try pw=12 as a middle ground between 10 and 15." The assistant was probing the memory ceiling empirically, trying to find the maximum viable parallelism. The pw=12 run was the third data point in this search (pw=10 ✓, pw=15 ✗, pw=12 ?).

What Came Next

The pw=12 failure triggered a deeper investigation. In the immediate next message ([msg 2984]), the assistant diagnosed: "OOM again at pw=12. The memory pressure is too high with 12 concurrent partition synthesis workers when you also have 15 concurrent jobs." The user then asked a pivotal question: "Maybe we're not freeing b_g2_m mem?" ([msg 2986]).

This question launched the investigation documented in chunk 1 of segment 30 — a deep dive into memory management that would ultimately reveal a subtle bug where the partition semaphore was releasing too early, allowing synthesized partitions to pile up in memory while waiting for the GPU channel. The fix — holding the semaphore permit until the job was delivered to the GPU channel — would reduce peak RSS from 668 GiB to 294.7 GiB, enabling pw=12 to run without OOM for the first time, albeit with a throughput regression that required further tuning.

The Deeper Lesson

This message, in its truncated, crash-marred output, encapsulates a fundamental tension in high-performance computing: the trade-off between parallelism and memory pressure. The Phase 12 split API was designed to improve throughput by decoupling the GPU worker from post-processing. But it also increased the system's appetite for parallelism — and therefore for memory. The memory ceiling, not the compute ceiling, became the binding constraint.

For the optimization project as a whole, this message marked a turning point. Before it, the team was focused on compute optimizations: faster kernels, better pipelining, reduced latency. After it, memory management became the central concern. The investigation that followed — building a global buffer tracker with atomic counters, tracing the lifetime of every large allocation, and redesigning the semaphore logic — was a direct consequence of the failure captured in this single benchmark run.

In the end, the pw=12 benchmark did not deliver the hoped-for throughput improvement. But it delivered something more valuable: a clear diagnosis of the system's true bottleneck, and the impetus to build the instrumentation needed to understand and fix it.