The Asymmetry of Memory: Why Pre-Allocation Failed to Move the Needle

A Performance Investigation That Defied Intuition

In the high-stakes world of Filecoin proof generation, every second counts. The SUPRASEAL_C2 Groth16 proving pipeline, responsible for producing the cryptographic proofs that underpin Filecoin's storage verification, had just achieved a significant victory: by moving synchronous destructor calls to background threads, the team had reclaimed 10 seconds from the GPU wrapper phase. But this success raised a natural question from the user — if deallocation was costing 10 seconds, shouldn't allocation be costing a similar amount? The assistant's response, message 1350 in the conversation, delivers the answer to that question, and it is a masterclass in the importance of measurement over intuition.

The Subject Message

Here is the message in full:

### Summary

>

Your intuition was sound — if dealloc was 10s, alloc should be comparable. The investigation revealed:

>

Dealloc (10s): The 10s was NOT from allocation/deallocation of the synthesis Vecs. It was from the C++ split_vectors + tail_msm_bases (~37 GB) and Rust ProvingAssignment (~130 GB) being freed synchronously after GPU proving completes. These are the GPU-phase working data, not synthesis data. The fix (async dealloc) moves the free to a background thread.

>

Alloc during synthesis: The synthesis Vecs also grow to ~130 GB total, but: - They grow incrementally via push() with geometric doubling — amortized O(1) - Growth is interleaved with computation (field arithmetic dominates at 16.98% per the perf profile) - 10 circuits grow in parallel across 96 cores — no serialization - Result: pre-allocation saves 0.0s

>

The asymmetry makes sense: dealloc was costly because munmap() of large contiguous regions is a synchronous kernel operation that tears down page tables all at once. Alloc via push() spreads the cost across millions of operations, each touching only a page or two.

>

Current standings: 77.0s total (50.8s synth + 26.2s GPU). The synthesis bottleneck at ~50.8s is now dominated by pure computation (field arithmetic + LC evaluation), not memory management. Further gains require Phase 5 (PCE) to eliminate synthesis entirely.

The Context: A Pipeline Under Optimization

To understand this message, one must appreciate the journey that led to it. The SUPRASEAL_C2 pipeline is a beast: it synthesizes Groth16 proofs for Filecoin's Proof-of-Replication (PoRep), consuming approximately 200 GiB of peak memory across its CPU synthesis and GPU proving phases. Over the course of the project, the team had already implemented Phase 1 (batch-mode pipeline rewrite), Phase 2 (async overlap between synthesis and GPU proving), Phase 3 (cross-sector batching), and Phase 4 (compute-level optimizations including Boolean::add_to_lc, Vec recycling pools, software prefetch, and CUDA timing instrumentation).

The most recent win had been the async deallocation optimization. After GPU proving completes, the pipeline frees massive data structures: approximately 37 GB of C++ vectors (split_vectors, tail_msm_bases) and roughly 130 GB of Rust Vecs (the ProvingAssignment a/b/c constraint vectors). Originally, these were freed synchronously — the GPU wrapper function would finish its CUDA work, then spend 10 seconds tearing down page tables via munmap() before returning. The fix was elegant: move these deallocations into detached threads on both the C++ and Rust sides, allowing the function to return immediately while the kernel reclaims memory in the background. This single change dropped the GPU wrapper time from 36.0s to 26.2s, matching the CUDA internal timing exactly.

The Hypothesis That Sparked the Investigation

The user's intuition was entirely reasonable. If freeing 167 GB of data took 10 seconds, then allocating that same 167 GB should also take time. The Rust Vec grows via push(), and behind the scenes, when capacity is exhausted, it must reallocate to a larger buffer, copy the old elements, and free the old allocation. For a Vec that grows from zero to 4 GB (the size of a single a vector), this involves approximately 27 doublings. Across 10 parallel circuits, that's 270 reallocation events, each copying progressively larger chunks of memory. The total volume of redundant copies was estimated at roughly 265 GB — a staggering amount of data movement.

If allocation overhead were truly comparable to deallocation, the team could expect to reclaim another 10 seconds simply by pre-allocating the Vecs to their final capacity, eliminating the reallocation churn entirely. This was a compelling hypothesis, and the assistant set out to test it.

The Investigation: Wiring Up a Hint System

The assistant discovered that bellperson, the underlying library, already had a SynthesisCapacityHint API designed for exactly this purpose. It allowed callers to specify the expected number of constraints and inputs before synthesis begins, enabling the Vecs to be pre-allocated to their final capacity. However, this API had never been wired up in the pipeline callers — every synthesis call was growing its Vecs organically.

The implementation was non-trivial. The assistant needed to:

  1. Create a global hint cache that stores the capacity measurements from the first synthesis of each circuit type
  2. Modify all six synthesis call sites in pipeline.rs to use the new synthesize_circuits_batch_with_hint function
  3. Handle all circuit variants (Porep32G, Porep64G, WindowPost32G, WinningPost32G, SnapDeals32G, SnapDeals64G)
  4. Ensure the hint cache was thread-safe and had zero overhead when no hint was available The code was written, compiled, and committed. The first synthesis of each circuit type would grow organically (as before), recording the final capacities. Subsequent syntheses would read the cached hint and pre-allocate, theoretically eliminating all reallocation overhead.

The Benchmark: Reality Intervenes

The assistant ran a rigorous two-phase benchmark. First, a single-partition microbenchmark (1 circuit, not the full 10) to isolate the effect:

Why the Asymmetry? A Deep Dive

The assistant's explanation reveals a fundamental truth about memory management in modern systems. The asymmetry between allocation and deallocation is not about the volume of data, but about the mechanics of how the operating system handles memory.

Deallocation via munmap() is a synchronous kernel operation. When a large memory-mapped region is freed, the kernel must walk the page tables, unmap every page, update the TLB, and mark the physical pages as free. For a 4 GB region, this involves millions of page table entries. The operation is inherently serial and CPU-bound within the kernel, and it blocks the calling thread until complete. This is why the original GPU wrapper spent 10 seconds in destructors — it was waiting for the kernel to finish tearing down page tables for ~167 GB of mapped memory.

Allocation via push() is fundamentally different. Rust's Vec uses geometric doubling (typically doubling capacity when full), which means the total cost of all reallocations is bounded by O(n) — each element is copied at most once on average. More importantly, the reallocations happen incrementally throughout the 50-second synthesis window. The first 20 doublings copy less than 256 MB total, happening when the Vecs are tiny. The late-stage doublings (from 2 GB to 4 GB) happen only 3-4 times per Vec, each copying 2-4 GB at memory bandwidth speeds (~100ms on a modern Zen4 system with ~40 GB/s bandwidth). These copies are interleaved with the actual computation — field arithmetic, linear combination evaluation, and constraint generation — which dominates the profile at 16.98% of CPU time.

Furthermore, the 10 circuits synthesize in parallel across 96 cores via rayon. This means the reallocation work is distributed across all available cores, and the memory copies overlap with computation happening on other cores. There is no serialization point where all cores must wait for a single reallocation to complete.

The assistant's calculation is illuminating: for a single circuit, the total wasted copy volume is approximately 26.5 GB (across all four Vecs: a, b, c, aux_assignment). At 40 GB/s memory bandwidth, that's about 0.66 seconds of pure copy time. Spread across 50.7 seconds of synthesis and interleaved with computation, it becomes invisible.

The Deeper Lesson: Measurement Over Intuition

This investigation embodies a principle that every performance engineer must internalize: intuition is a starting point, not a conclusion. The user's hypothesis was reasonable, well-motivated, and grounded in a correct observation (dealloc was expensive). The assistant's implementation was thorough, correct, and well-engineered. The benchmark was rigorous, testing at two different scales (single circuit and full batch). And the result was definitive: zero impact.

The 117 lines of code implementing the hint cache were not wasted. As the assistant noted, the infrastructure serves as "defensive code" — it prevents pathological fragmentation under memory pressure and costs nothing when not triggered. But the critical insight is that the synthesis bottleneck is now purely computational, not memory-management-related. The Vec reallocation overhead, while theoretically large in terms of bytes copied, is practically invisible because of how Rust's allocator amortizes cost and how modern CPU memory bandwidth overlaps with computation.

This finding has direct implications for the project's roadmap. The assistant concludes: "Further gains require Phase 5 (PCE) to eliminate synthesis entirely." PCE (Precomputed Circuit Evaluation) is a proposed optimization that would replace the entire CPU synthesis phase with precomputed values, effectively trading memory for CPU time. The 50.8 seconds of synthesis is now understood to be dominated by field arithmetic and linear combination evaluation — pure computation that cannot be eliminated by better memory management. Only by fundamentally changing the architecture (PCE) or by accelerating the computation itself (e.g., through GPU-based synthesis) can the next major improvement be achieved.

Conclusion

Message 1350 is a turning point in the optimization journey. It closes the book on memory-management optimizations for the synthesis phase and opens the door to Phase 5's architectural changes. The message is notable not just for its technical content but for its intellectual honesty — the willingness to invest effort in a promising hypothesis, to measure rigorously, and to accept a null result with grace and insight.

The asymmetry between allocation and deallocation is now understood at a deep level: munmap() is a synchronous kernel operation that blocks until page tables are torn down, while push() with geometric doubling is an amortized O(1) operation that overlaps with computation. The 10-second deallocation win was real and important. The 0-second allocation "win" is equally informative — it tells the team where not to look for further gains, focusing their efforts on the computational bottlenecks that remain.

In the end, the most valuable optimization is sometimes the one you don't make — because the knowledge gained from proving it unnecessary is worth more than the speedup you hoped for.