The Diagnostic Pivot: Recognizing a Memory Leak in Phase 12's Split GPU Proving API

Introduction

In the high-stakes world of Filecoin proof generation, every gigabyte of memory counts. The SUPRASEAL_C2 Groth16 proving pipeline consumes approximately 200 GiB at peak for a single proof, and the team behind it has been iterating through optimization phases to squeeze out every ounce of throughput. Phase 12 introduced a "split GPU proving API" that offloaded the b_g2_msm computation from the GPU worker's critical path, achieving a respectable 37.1 seconds per proof — a 2.4% improvement over the Phase 11 baseline. But when the team tried to push synthesis parallelism higher, they hit a wall: out-of-memory (OOM) errors at partition_workers=15 and even partition_workers=12. The natural assumption was that the split API was holding onto memory longer than before. But a single remark from the user — "It's not 400GB tho, something is leaking somewhere" — triggered a fundamental shift in the investigation. Message 2994 captures that pivot: the moment the assistant recognized that the OOM behavior was anomalous and reframed the problem from "memory pressure" to "memory leak."

The Message

The subject message reads:

You're right — pw=10 worked fine, and pw=12 only adds 2 more partitions at ~13 GiB each (~26 GiB extra). That shouldn't OOM on 755 GiB. Something is leaking. Let me check if the PendingProofHandle's Rust-side data is actually being freed after finalization, and whether the async dealloc threads are stacking up.

This is followed by a todo list update that marks the Phase 12 compilation fixes and pw=10 benchmark as completed, marks the pw=12/15 OOM as completed, and adds a new high-priority item: "Investigate memory leak in Phase 12 split API path."

Context: The Road to Phase 12

To understand the significance of this message, we need to trace the path that led to it. The team had been working through a series of optimization phases for the cuzk SNARK proving engine. Phase 11 had addressed DDR5 memory bandwidth contention with three targeted interventions, achieving a baseline of 38.0 seconds per proof. Phase 12 was designed to further improve throughput by decoupling the GPU worker's critical path from CPU post-processing — specifically, by splitting the b_g2_msm computation into a background task that could run asynchronously while the GPU worker picked up the next partition.

The split API was implemented across multiple layers: C++ CUDA code in groth16_cuda.cu, Rust FFI bindings in supraseal.rs, and the engine worker loop in engine.rs. After fixing a series of compilation errors — a missing SynthesisCapacityHint struct, an unused generic parameter, a use-after-free bug where a background thread captured a dangling reference to a stack-allocated array — the implementation was successfully committed to the feat/cuzk branch.

Benchmarking with the optimal configuration (gw=2, pw=10, gt=32, j=15) yielded 37.1 seconds per proof. This was a tangible improvement, but the team wanted more. The user suggested increasing synthesis parallelism: "Try higher synthesis partition_workers in config, maybe 15/20?" The logic was sound — if the split API freed the GPU worker faster, the bottleneck might shift to synthesis throughput, and more partition workers could keep the GPU better fed.

The OOM Anomaly

The results were immediate and alarming. At pw=15, the daemon OOM'd before completing a single proof. At pw=12, it managed eight proofs before crashing. The initial suspicion, voiced by the user in message 2986, was that the split API was not freeing b_g2_msm memory properly — that the background thread was holding onto GPU resources that should have been released. The assistant began investigating this angle.

But then the user reconsidered. In message 2990, they said: "It's not 400GB tho." And in message 2992 (implied by the context), they refined this to: "It's not 400GB tho, something is leaking somewhere."

This was the crucial insight. The assistant's response in message 2994 shows the moment of recognition. Let's do the math: pw=12 adds only 2 more partition workers compared to pw=10. Each partition synthesis uses approximately 13 GiB. That's 26 GiB of additional memory pressure. On a system with 755 GiB of RAM, that should be trivially absorbable. Even accounting for the SRS (44 GiB), PCE (26 GiB), parsing overhead, and other fixed costs, the total should be well within the memory budget. The OOM at pw=12 is not a capacity problem — it's a leak.

The Reasoning and Motivation

Why was this message written? The assistant had been operating under the assumption that the OOM at higher partition_workers was a straightforward capacity issue — too many concurrent syntheses consuming too much RAM. The user's correction forced a re-evaluation of that assumption. The assistant performs a quick mental calculation: 2 partitions × 13 GiB = 26 GiB extra. On 755 GiB total. That doesn't add up to an OOM. Therefore, something else must be going on.

The motivation is diagnostic precision. The assistant is acknowledging the user's insight and pivoting the investigation from "how do we reduce memory pressure" to "where is the leak." This is a critical shift because it changes the entire investigative approach. Instead of tuning parameters and reducing concurrency, the team needs to trace memory allocation and deallocation through the entire pipeline — specifically the lifecycle of PendingProofHandle and the behavior of async deallocation threads.

Decisions Made

This message doesn't contain tool calls or code changes, but it makes several important decisions:

1. Accept the user's hypothesis: The assistant explicitly agrees ("You're right") and adopts the leak hypothesis as the new working theory.
2. Identify specific investigation targets: Two concrete areas are identified for inspection — whether PendingProofHandle's Rust-side data is actually freed after finalization, and whether async dealloc threads are stacking up (i.e., not completing their cleanup before new allocations are made).
3. Update the todo list: The investigation is formally recorded with a new high-priority todo item, signaling a change in direction to anyone reading the conversation.
4. Abandon the capacity-tuning approach: Instead of trying pw=11 or other intermediate values, the assistant commits to finding the root cause of the leak.

Assumptions

The message rests on several assumptions:

• Partition memory footprint is ~13 GiB: This is based on earlier analysis of the synthesis pipeline. It's a well-supported estimate from the team's prior work.
• The system has 755 GiB of usable RAM: This is the physical capacity of the machine. The assumption is that the OS and other processes don't consume a significant fraction of this.
• The leak is in the Rust-side data lifecycle: The assistant specifically targets PendingProofHandle and async dealloc threads. This assumes the C++ side (GPU memory) is correctly managed, which is reasonable given that the C++ code had already been audited for the use-after-free bug.
• The leak is cumulative: The fact that pw=12 runs for 8 proofs before OOM'ing suggests a gradual accumulation of unreleased memory, not a single large allocation that exceeds the budget.

Mistakes and Incorrect Assumptions

The primary mistake was the initial assumption that the OOM was a straightforward capacity issue. The assistant's first reaction to the pw=15 OOM was to try pw=12 as a middle ground, and when that also OOM'd, the initial hypothesis was that the split API was holding onto b_g2_msm memory. The user's intervention was necessary to recognize that the magnitude of the OOM was inconsistent with the expected memory footprint.

This is a common cognitive trap in performance debugging: when a system crashes under increased load, the natural instinct is to attribute it to resource exhaustion rather than a bug. The assistant fell into this trap, and the user pulled them out of it.

There's also an implicit assumption that the async deallocation mechanism works correctly. The assistant had implemented early deallocation of NTT evaluation vectors in prove_start and built a global buffer tracker with atomic counters. But the question of whether PendingProofHandle is properly freed after finalization — and whether the async finalizer tasks complete in a timely manner — had not been thoroughly verified.

Input Knowledge Required

To understand this message, one needs:

• The Phase 12 split API design: Knowledge that the split API offloads b_g2_msm to a background thread while the GPU worker moves on, and that this creates a PendingProofHandle that holds Rust-side synthesis data until finalization completes.
• Memory accounting from earlier phases: The ~13 GiB per partition estimate, the ~44 GiB SRS, ~26 GiB PCE, and other fixed costs that were documented in the background reference.
• The system's hardware configuration: 755 GiB of RAM, which was established earlier in the conversation.
• The async deallocation mechanism: The fact that finalization runs in spawned async tasks, and that these tasks might stack up if they don't complete before new ones are created.
• The benchmark results: pw=10 works (37.1s/proof), pw=15 OOM immediately, pw=12 OOM after 8 proofs.

Output Knowledge Created

This message creates several important pieces of knowledge:

1. The OOM at pw=12 is anomalous: A formal recognition that the expected memory pressure from 2 extra partitions (~26 GiB) cannot explain an OOM on a 755 GiB system.
2. A new investigation hypothesis: "Something is leaking" — specifically in the PendingProofHandle lifecycle or async deallocation thread management.
3. Specific investigation targets: Two concrete areas to examine — PendingProofHandle deallocation and async dealloc thread stacking.
4. A documented shift in direction: The todo list update serves as a record that the team is now pursuing a memory leak investigation rather than a capacity tuning approach.

The Thinking Process

The thinking visible in this message is concise but revealing. The assistant performs a rapid back-of-the-envelope calculation: pw=12 adds 2 partitions, each ~13 GiB, total ~26 GiB extra. The system has 755 GiB. The conclusion is immediate: "That shouldn't OOM on 755 GiB." The word "shouldn't" is telling — it indicates a discrepancy between expectation and observation, which is the hallmark of a bug rather than a capacity limit.

The assistant then proposes two specific investigative avenues. The first — checking if PendingProofHandle's Rust-side data is actually freed after finalization — targets the core lifecycle of the split API. If the handle holds onto the provers, input_assignments, and aux_assignments vectors after finalization completes, those ~13 GiB per partition would never be released, accumulating with each proof until the system runs out of memory.

The second — checking if async dealloc threads are stacking up — targets a more subtle issue. If the finalizer tasks are spawned faster than they complete, they could accumulate in the async runtime's task queue, each holding its own memory reservation. This would manifest as a gradual memory growth over time, consistent with the pw=12 behavior where OOM occurs after 8 proofs rather than immediately.

Conclusion

Message 2994 is a classic diagnostic pivot. It's the moment when a team stops trying to squeeze more performance out of a system and starts asking whether the system is fundamentally broken. The assistant's willingness to accept the user's correction, perform the mental calculation, and redirect the investigation is a model of collaborative debugging. The message is short — barely a paragraph — but it encapsulates a critical insight that would shape the next phase of work. The leak investigation that followed would lead to the discovery of the partition semaphore issue, the global buffer tracker instrumentation, and the eventual resolution of the memory buildup problem. But it all started with this moment of recognition: "Something is leaking."