From 200 GiB to a Continuous Pipeline: Architecting the cuzk SNARK Proving Daemon

Introduction

In the world of Filecoin storage proving, few numbers capture the imagination quite like 200 GiB. That is the peak memory footprint of SUPRASEAL_C2, the Groth16 proof generation pipeline at the heart of Filecoin's Proof-of-Replication (PoRep) system. Every time a storage provider needs to generate a proof—proving they are still storing the data they committed to—the system balloons to consume nearly a quarter of a terabyte of GPU and host memory. On cloud rental markets where memory is the dominant cost driver, this makes each proof eye-wateringly expensive.

But the 200 GiB number is not just a cost problem. It is a symptom of deeper architectural decisions: loading all ten partition circuits in parallel, keeping the entire Structured Reference String (SRS) pinned in GPU memory, and treating each proof job as an isolated batch process with no state carried between invocations. The investigation documented in this chunk set out to understand every byte of that 200 GiB, map the full call chain from Curio's Go task layer through Rust FFI into C++/CUDA kernels, and then—crucially—to think bigger. Not just to optimize individual steps, but to re-architect the entire proving pipeline into a continuous, memory-efficient system designed for heterogeneous cloud GPU fleets.

This article synthesizes that investigation. It covers the comprehensive mapping of the existing pipeline, the nine structural bottlenecks identified, the three optimization proposals that emerged, and the micro-optimization analysis that examined CPU and GPU hotpaths at the instruction level. Most importantly, it traces the arc from analysis to design: how the team moved from understanding what exists to imagining what could be built—a pipelined SNARK proving daemon called cuzk.

The Landscape: Understanding the 200 GiB Beast

The investigation began where all good systems work begins: with a map. The SUPRASEAL_C2 pipeline is not a single program but a chain of components spanning multiple languages and runtime environments. At the top sits Curio, a Go-based task orchestrator that manages storage proving workloads across a fleet of machines. When a proof is needed, Curio spawns a child process via an ffiselect mechanism—a Rust-based selector that decides which proving backend to use based on the proof type and available hardware. That Rust layer invokes bellperson, a Rust library for Groth16 proof generation, which in turn calls into supraseal-c2, a C++ library with CUDA kernels that perform the heavy lifting: multi-scalar multiplication (MSM), number-theoretic transforms (NTT), and constraint evaluation.

The call chain is deep, but the memory accounting is surprisingly straightforward. The ~200 GiB peak breaks down as follows: ten partition circuits, each consuming approximately 16 GiB of GPU memory, run in parallel. On top of that, approximately 48 GiB of SRS parameters are loaded into pinned host memory and transferred to the GPU. Add intermediate buffers for polynomial evaluations, MSM operands, and NTT twiddle factors, and the total reaches the 200 GiB mark.

But why ten partitions in parallel? The answer lies in the architecture of Groth16 proof generation for Filecoin's sector size. A single PoRep proof requires evaluating a circuit over ten partitions of the sector data. The existing implementation launches all ten partitions simultaneously, each as a separate CUDA stream, and waits for all to complete before assembling the final proof. This parallelism maximizes GPU utilization for a single proof but comes at the cost of massive peak memory—each partition needs its own copy of the circuit evaluation state, its own intermediate buffers, and its own share of the SRS.

The investigation catalogued this and eight other structural bottlenecks in a comprehensive background reference document. Beyond the parallel partition problem, the document identified: the SRS loading overhead (approximately 60 seconds per proof job, dominated by disk I/O and GPU transfer), the lack of state persistence between proof jobs (every invocation is a cold start), the absence of pipelining between proof phases (witness generation, MSM, NTT, and commitment assembly run sequentially with no overlap), the coarse-grained scheduling (proofs are scheduled as monolithic units rather than at phase boundaries), the homogeneous resource model (all GPUs treated identically regardless of memory capacity), the lack of preemption support (a long-running proof blocks the entire GPU), the fixed batch size (no dynamic batching across proof types), and the absence of graceful degradation (if memory is insufficient, the proof simply fails).

This document became the foundation for everything that followed. It was not merely a catalog of problems but a map of opportunities—each bottleneck was a target for optimization.

Thinking Bigger: From Optimization to Architecture

The initial optimization proposals were conservative: reduce peak memory by streaming partitions sequentially rather than in parallel (Sequential Partition Synthesis), eliminate SRS loading overhead by keeping the proving process alive across proofs (Persistent Prover Daemon), and improve throughput by batching multiple sectors' circuits into single GPU invocations (Cross-Sector Batching). These three proposals, taken together, projected a reduction in peak memory from ~200 GiB to approximately 64–103 GiB and a 2–3× improvement in throughput per GPU.

But the user challenged the team to think bigger. Instead of optimizing individual proof generation steps, the goal became to architect an entirely new system: a continuous, memory-efficient proving pipeline optimized for heterogeneous cloud rental markets. This was not a local optimization problem; it was a systems architecture problem. And systems architecture problems of this kind—managing scarce GPU memory, scheduling heterogeneous workloads across multiple devices, handling graceful degradation under resource constraints—had been solved, at enormous scale and sophistication, by the machine learning inference community.

This realization triggered a deliberate methodological pivot. The team turned their attention away from the SUPRASEAL_C2 codebase and toward GPU inference engines: vLLM, TensorRT-LLM, Triton Inference Server, and llama.cpp. The insight was that model weights in an LLM are structurally analogous to SRS parameters in a proving system, inference requests are analogous to proof jobs, and KV caches are analogous to intermediate witness vectors. If inference engines had solved the problem of managing multiple models with limited GPU memory, scheduling requests across heterogeneous hardware, and gracefully degrading under memory pressure, those solutions could be translated into the proving domain.

Architecture Mining: What Inference Engines Taught Us

The research into inference engines proceeded in two waves. The first wave [1] cast a broad net with six parallel searches covering model loading/unloading, PagedAttention memory management, TensorRT-LLM multi-model serving, GPU inference scheduling, Triton model management, and llama.cpp server mode. The results were informative but incomplete—the assistant needed to go deeper on specific mechanisms.

The second wave [2] was a targeted follow-up. Four specific searches probed NVIDIA Run:ai's GPU memory swap for hot-swapping patterns, vLLM's continuous batching scheduler architecture at the iteration level, Triton's rate limiter and instance groups for resource governance, and graceful degradation strategies for heterogeneous GPU memory sizes. This two-wave pattern—broad then targeted—proved highly effective at surfacing the most relevant architectural patterns.

The synthesis of this research [3] produced a comprehensive structured analysis of six architectural domains, each mapped to a concrete proving engine analogy. The key insights were:

Memory Management: vLLM's PagedAttention demonstrated that GPU memory could be divided into fixed-size blocks with virtual-to-physical translation, achieving near-zero fragmentation (<4%) compared to the 60–80% waste of naive pre-allocation. For the proving engine, this meant pre-allocating GPU memory into fixed-size blocks for SRS segments and intermediate proof values, with a block table tracking ownership and immediate reclamation on proof completion.

Model Loading/Unloading: vLLM's Sleep Mode introduced a two-tier hibernation system (Level 1: offload to CPU RAM for fast wake; Level 2: discard entirely for near-zero RAM usage) that was 18–200× faster than full reload because it preserved CUDA contexts, memory allocations, and communication channels. For the proving engine, this translated to a three-tier SRS hierarchy: hot (currently-active circuit params on GPU), warm (recently-used params pinned in host RAM for fast PCIe transfer), and cold (on-disk, loaded on demand).

Request Scheduling: Continuous batching at the iteration level—where new requests can join a running batch at any iteration boundary and completed requests leave immediately—was identified as the single most important innovation in modern inference serving, delivering 2–24× throughput improvements over static batching. For the proving engine, this meant scheduling at proof-phase boundaries rather than whole-proof boundaries, allowing interleaving of different proof types' phases to maximize GPU utilization.

Pipeline Parallelism: The disaggregated prefill/decode paradigm from vLLM and the pipelined-ring parallelism from prima.cpp inspired a three-track pipeline for the proving engine: SRS loading on DMA, witness generation on CPU, and MSM/NTT computation on GPU, all running concurrently across different proofs.

Multi-Model Serving: Triton's instance groups and rate limiter provided a model for declaring resource requirements per circuit type and preventing oversubscription. MIG partitioning on A100/H100 GPUs offered hardware-level isolation for running different proof types on the same GPU.

Graceful Degradation: Research systems like Hetis, SchedTune, and CARMA demonstrated memory-aware placement with dynamic adaptation to heterogeneous GPU fleets, achieving 81% higher GPU memory utilization and 100% OOM avoidance.

The cuzk Architecture: From Patterns to Design

The inference engine patterns were not merely catalogued—they were synthesized into a concrete architecture for the cuzk proving daemon [4]. The architecture has five layers:

Request Router: Routes proof requests by type to the appropriate scheduler, with priority queues for different proof types (PoRep, SnapDeals, WindowPoSt, WinningPoSt).
Global Scheduler: Implements continuous batching at phase boundaries, circuit-aware placement (matching proof types to GPUs with sufficient memory), priority-based scheduling, and preemption support.
Block Memory Manager: Implements a PagedAttention-inspired fixed-size block pool with O(1) allocation/deallocation, a two-tier GPU/host RAM hierarchy, and async DMA offload/prefetch on dedicated CUDA streams.
Circuit/SRS Manager: Implements the three-tier (hot/warm/cold) hierarchy with explicit load/unload API, sleep/hibernate for fast reactivation, LRU eviction, and progressive streaming from storage.
Pipeline Executor (per GPU): Implements a four-phase pipeline (witness prep → MSM/NTT → commitment → assembly) with phase overlap across different proofs, SRS prefetching, and immediate block release on phase completion. The architecture was designed with pragmatic incrementalism in mind. Phase 0 requires zero upstream modifications to the existing SUPRASEAL_C2 libraries—it simply wraps the existing child process model with a persistent daemon that keeps SRS resident in memory across proof jobs, delivering an immediate +25% throughput improvement. Subsequent phases add multi-type scheduling, pipelining, batching, compute optimizations, and finally the pre-compiled constraint evaluator (PCE), with cumulative throughput gains projected at 1.3× to 10×+ over the 18-week roadmap.

Micro-Optimization Analysis: The Devil in the Details

Alongside the architectural work, a parallel thread of micro-optimization analysis examined the computational hotpaths at the instruction level. The CPU synthesis hotpath—the enforce() loop that evaluates circuit constraints—was analyzed for SHA-256 bit manipulation patterns, Fr field arithmetic, and memory access patterns. The analysis found that the constraint evaluation is dominated by boolean operations on the ~99% boolean aux_assignment vector, suggesting opportunities for SIMD vectorization and bit-packing.

On the GPU side, the NTT and MSM kernels were characterized in terms of memory bandwidth utilization, occupancy, and compute-to-global-load ratios. The NTT kernels were found to be memory-bandwidth-bound, while MSM kernels were compute-bound with significant shared memory pressure. The H-to-D transfer patterns revealed that SRS loading and intermediate result transfer could be overlapped with computation using multiple CUDA streams—a key enabler for the pipelined architecture.

Perhaps the most intriguing micro-optimization finding was the feasibility of recomputing a/b/c vectors on-the-fly rather than materializing them in memory. The a/b/c vectors—the three wire assignments in the R1CS constraint system—account for a significant fraction of the intermediate buffer footprint. The analysis showed that these vectors could be recomputed from the witness data in approximately the same time as reading them from memory, suggesting that the memory savings (potentially tens of GiB) could be achieved at negligible computational cost.

The Bigger Picture: A New Paradigm for Proving

The overarching achievement of this investigation is a shift in perspective. The team began by asking "how do we make each proof generation step faster?" and ended by asking "how do we architect a continuous proving pipeline that treats memory as the primary constraint?" This is not a minor reframing—it is a paradigm shift.

The existing SUPRASEAL_C2 system treats proof generation as a batch process: load everything, compute everything, return the result, tear everything down. The cuzk architecture treats proof generation as a continuous pipeline: keep SRS resident, overlap phases across proofs, schedule at phase boundaries, and reclaim memory the instant it is no longer needed. This is the difference between a batch-oriented mainframe and a modern multitasking operating system—the same architectural leap that transformed computing in the 1960s, now applied to cryptographic proving.

The implications extend beyond Filecoin. The patterns discovered here—block-based memory management, tiered parameter storage, iteration-level scheduling, disaggregated pipeline parallelism—are applicable to any GPU-accelerated cryptographic workload. Fully homomorphic encryption, zero-knowledge virtual machines, and verifiable computation all face the same fundamental challenge: large parameters, multi-phase computation, and the need to run efficiently on heterogeneous hardware. The cuzk architecture provides a template for how to build such systems.

Conclusion

The investigation documented in this chunk represents a comprehensive journey from analysis to architecture. It began with a deep-dive into the SUPRASEAL_C2 pipeline, producing a detailed map of the call chain, memory accounting for the ~200 GiB peak footprint, and identification of nine structural bottlenecks. It expanded through three optimization proposals that promised meaningful but incremental improvements. And it culminated in a bold re-architecting of the entire proving pipeline, inspired by patterns from GPU inference engines and synthesized into the cuzk proving daemon architecture.

The work demonstrates that the hardest problems in cryptographic proving are not cryptographic at all—they are systems problems. Memory management, scheduling, pipelining, and graceful degradation are challenges that other domains have already solved at massive scale. The task is not to invent new solutions but to recognize the patterns and translate them correctly. The cuzk architecture is the result of that translation: a continuous, memory-efficient proving pipeline that treats the GPU not as a batch accelerator but as a shared resource to be managed with the same sophistication that inference engines have brought to LLM serving.

For the Filecoin ecosystem, the promise is dramatic: a 5–6× reduction in cost per proof, the ability to run on cheaper, heterogeneous GPU hardware, and a proving pipeline that can scale with the network's growth. For the broader field of cryptographic computing, the architecture provides a blueprint for how to build systems that are not just correct but efficient—systems that treat memory as the scarce resource and design every layer around its conservation.## References

[1] "Bridging Worlds: How GPU Inference Engine Architecture Informs the Design of a SNARK Proving Daemon" — Analyzes the initial research brief that launched the investigation into inference engine patterns, examining the six targeted questions and their mapping to proving engine challenges.

[2] "Architecture Mining: How GPU Inference Engines Inform the Design of a Proving Daemon" — Documents the first wave of six parallel web searches across vLLM, TensorRT-LLM, Triton, and llama.cpp, capturing broad architectural patterns.

[3] "The Second Pass: How Targeted Web Searches Refined a GPU Proving Engine Architecture" — Describes the targeted second wave of searches that filled specific gaps on Run:ai memory swap, vLLM scheduler internals, Triton rate limiter, and graceful degradation strategies.

[4] "Architecture Transfer: How GPU Inference Engine Patterns Are Reshaping SNARK Proof Generation" — Presents the comprehensive synthesis of inference engine patterns into the five-layer cuzk architecture, with detailed analogies across six architectural domains.