The Architecture of a SNARK Proving Engine: Designing cuzk — A Pipelined, GPU-Resident Proof Generation Daemon for Filecoin

Introduction

On a cold Linux machine in the Curio codebase repository, an extraordinary act of systems architecture unfolded. The assistant, having spent the preceding nine messages exhaustively mapping the entire Groth16 proof generation pipeline for Filecoin's storage proof system, sat down to design a new piece of infrastructure: a persistent, GPU-resident SNARK proving daemon called cuzk. The resulting message — message 77 in the conversation — is not merely a plan. It is a masterclass in applied systems thinking, drawing inspiration from GPU inference engines, deeply understanding the memory hierarchy of zero-knowledge proofs, and weaving together half a dozen prior optimization proposals into a coherent, phased architecture.

This article examines that single message in microscopic detail. We will explore why it was written, what decisions it encodes, what assumptions underpin it, where it might be wrong, and what knowledge it both consumes and produces. The message is a turning point in the conversation: the moment when analysis crystallizes into design, when understanding the problem becomes planning the solution.

Context: The Conversation That Led to This Moment

To understand message 77, we must first understand what came before it. The conversation spans 77 messages total, with message 77 being the final assistant response in a long chain of investigation. The user's initial request (message 67) was deceptively simple: "look at all proposals, plan a pipelined snark daemon which accepts a pipeline (some rpc) of PoRep/SnapDeals/PoSt snarks (also on various sector sizes), schedules the work, and outputs results."

But this simple request landed on a foundation of extraordinary depth. The conversation's earlier segments (segments 0 through 2) had already produced:

The Structure of the Message: An Architecture Document in Miniature

The message is structured as a complete architecture document. It contains 13 major sections, each addressing a different aspect of the proposed system:

  1. Vision & Positioning — The "why" of cuzk, with explicit analogies to inference engines
  2. Architecture Overview — A high-level diagram and component description
  3. Proof Types & Circuit Profiles — Resource requirements for each proof type
  4. gRPC API Surface — The complete protobuf schema
  5. SRS Memory Manager — Tiered Residency — The hot/warm/cold memory hierarchy
  6. Scheduler Design — Priority queues, batch accumulation, GPU affinity
  7. GPU Worker Pipeline — The per-GPU execution loop
  8. Library / Binary Design — The Rust crate structure
  9. Phased Implementation Roadmap — 18 weeks across 6 phases
  10. Configuration — The TOML config file schema
  11. Test Harness — Golden test data and test modes
  12. Key Design Decisions & Tradeoffs — A decision matrix
  13. Open Questions — Four unresolved issues This structure itself reveals the assistant's thinking process. It starts with vision (the highest level of abstraction), drills down into architecture and components, then produces a concrete implementation plan with phases and timelines. It ends with tradeoffs and open questions — acknowledging uncertainty. This is the structure of a seasoned systems architect who knows that the value of a design document lies not in being right about everything, but in being explicit about everything. Let us now examine each section in depth, extracting the reasoning, assumptions, decisions, and knowledge embedded within.

Section 1: Vision & Positioning — "vLLM for Zero-Knowledge Proofs"

The message opens with a bold analogy: "cuzk is a persistent GPU-resident SNARK proving engine — think 'vLLM for zero-knowledge proofs.'" This single sentence reveals the assistant's entire conceptual framework. The assistant has spent significant effort studying GPU inference engine architectures (documented in message 68's task results), and is now explicitly mapping inference concepts onto proof generation concepts.

The table of analogies is worth quoting in full:

| Inference Concept | cuzk Equivalent | |---|---| | Model weights | SRS / Groth16 params (~47 GiB for PoRep, ~1 GiB for PoSt) | | Model loading/swapping | SRS loading/eviction from pinned memory | | Inference request | Proof request (C1 output + proof type + sector ID) | | KV cache / activations | Witness vectors, a/b/c evaluations, NTT intermediates | | Batch scheduling | Cross-sector proof batching (same circuit topology) | | Prefill vs decode | Synthesis (CPU) vs GPU compute (NTT/MSM) | | Model serving | Circuit-type serving (PoRep, WindowPoSt, WinningPoSt, SnapDeals) |

This mapping is not superficial. Each analogy is structurally precise:

The Reasoning Behind the Vision

Why did the assistant reach for inference engines as the mental model? The answer lies in the prior investigation. The assistant had just read about vLLM's sleep mode, Triton's model management, and TensorRT-LLM's batching strategies. It had also mapped the existing proof generation pipeline and identified that the fundamental problem was not computational speed but resource management: SRS loading dominated latency, memory fragmentation limited throughput, and the lack of persistent state meant every proof started from scratch.

The inference engine analogy provided a ready-made solution space. Instead of inventing a new resource management strategy from scratch, the assistant could adapt proven patterns from a mature ecosystem. This is a hallmark of good systems design: recognizing isomorphic problems and transferring solutions across domains.

Section 2: Architecture Overview — The Component Diagram

The architecture diagram in the message shows a layered system:

Curio (Go) → gRPC → cuzk daemon (Rust, tokio)
  ├── gRPC Server (tonic)
  ├── Scheduler
  │   ├── Priority queues per proof type
  │   ├── Batch collector
  │   ├── GPU affinity tracking
  │   ├── Deadline-aware scheduling
  │   └── Memory budget enforcement
  ├── GPU Workers (one per GPU)
  │   ├── SRS Manager (pinned memory)
  │   ├── Synthesis (CPU pool)
  │   └── NTT+MSM (CUDA)
  └── SRS Memory Manager (global)
      ├── Hot: CUDA pinned host RAM
      ├── Warm: Regular heap (mmap)
      └── Cold: Disk

This diagram encodes several design decisions:

Decision 1: Rust with tokio, not Go. The assistant chooses Rust as the implementation language, with the tokio async runtime. The rationale is stated in the tradeoffs table: "Direct access to bellperson/supraseal, no CGO overhead for hot path, better memory control." This is a significant decision because Curio itself is written in Go, and the existing ffiselect layer is Go. Choosing Rust means the daemon is a separate process, not a Go library. This has implications for deployment, debugging, and the skill set required.

Decision 2: gRPC over Unix socket, not shared memory. The assistant explicitly considered and rejected shared memory for the C1 output transfer. The reasoning is that at ~50 MB per PoRep C1 output, streaming over gRPC on a unix socket takes only ~5ms — well within acceptable latency. This avoids the complexity of shared memory management, reference counting, and lifecycle synchronization.

Decision 3: Separate scheduler and worker components. The scheduler is responsible for queue management, batching, and GPU affinity. The workers are responsible for actual proof execution. This separation allows the scheduler to make decisions based on global state (all queues, all GPUs, all SRS loads) while workers focus on efficient execution.

Decision 4: Global SRS memory manager, not per-GPU. The SRS manager is a global component that tracks all pinned memory across all GPUs. This is necessary because SRS files are large (~47 GiB for PoRep) and may need to be shared across GPUs or evicted when switching proof types. A per-GPU manager would lead to duplication and fragmentation.

Section 3: Proof Types & Circuit Profiles — The Resource Matrix

The message presents a table of resource requirements for each proof type:

| Proof Type | Constraints | FFT Domain | SRS Size | RAM/Proof | GPU VRAM | Partitions | Priority | |---|---|---|---|---|---|---|---| | PoRep C2 (32G) | ~130M | 2^27 | ~47 GiB | ~12-16 GiB | ~12-16 GiB | 10 | Normal | | PoRep C2 (64G) | ~130M | 2^27 | ~47 GiB | ~12-16 GiB | ~12-16 GiB | 10 | Normal | | SnapDeals (32G) | ~81M | 2^27 | ~30 GiB* | ~8-12 GiB | ~8-12 GiB | 16 | Normal | | WindowPoSt (32G) | ~125M | 2^27 | ~1-2 GiB | ~15-20 GiB | ~10-15 GiB | 1/part | High | | WinningPoSt (32G) | ~3.5M | 2^22 | ~200 MB | ~0.5-1 GiB | ~0.5 GiB | 1 | Critical |

This table is the output of the assistant's prior investigation into circuit sizes and resource profiles (messages 68, 70, 75). It encodes several critical insights:

Insight 1: The asymmetry of SRS sizes. PoRep and SnapDeals require ~30-47 GiB of pinned SRS data, while PoSt types require only ~200 MB - 2 GiB. This asymmetry drives the entire memory management strategy. On a machine with 96 GiB of RAM, you can keep one large SRS hot plus all PoSt SRS files simultaneously. On a machine with 256 GiB, you can keep everything hot.

Insight 2: WinningPoSt is tiny but critical. WinningPoSt has only ~3.5M constraints, an SRS of ~200 MB, and requires only ~0.5-1 GiB of RAM. But it has CRITICAL priority because it must complete within a Filecoin epoch (30 seconds) to avoid penalties. This means the scheduler must be able to jump a WinningPoSt proof to the front of the queue, potentially preempting a PoRep proof that has been running for minutes.

Insight 3: WindowPoSt has the highest per-proof RAM requirement. Despite having a small SRS, WindowPoSt requires ~15-20 GiB of working RAM per proof. This is because WindowPoSt proves over many sectors simultaneously, generating a large witness. This means WindowPoSt cannot be batched as aggressively as PoRep.

Insight 4: PoRep 32G and 64G have identical circuit sizes. The assistant notes that both sector sizes use ~130M constraints and the same 2^27 FFT domain. This is a subtle but important point: the circuit size depends on the proof type, not the sector size (for PoRep). This means the SRS is the same for both sector sizes, and batching can mix 32G and 64G proofs.

The asterisk on SnapDeals' SRS size (~30 GiB*) signals uncertainty. The assistant is noting that this is an estimate, not a confirmed value. This is honest about the limits of the investigation.

Section 4: The gRPC API Surface — Designing for Asynchrony

The protobuf schema in the message defines seven RPC methods:

  1. SubmitProof — Submit a proof request, return immediately with a job ID
  2. AwaitProof — Block until a submitted proof completes
  3. Prove — Submit + Await in one call (convenience)
  4. CancelProof — Cancel a pending or in-progress proof
  5. GetStatus — Query daemon capabilities and status
  6. GetMetrics — Prometheus-compatible metrics
  7. PreloadSRS / EvictSRS — Explicit SRS lifecycle management The Submit/Await split is the most important design decision in this API. It allows Curio to submit multiple proofs asynchronously, then await them independently. This enables the scheduler to reorder and batch proofs without blocking the submitter. The Prove method is a convenience wrapper for simple cases, but the real power is in the split API. The assistant's commentary on the API design reveals the reasoning:
- Submit/Await split: Allows Curio to submit multiple proofs, then await them independently. The scheduler can reorder/batch without blocking the submitter. - Prove (unary): Convenience for simple cases — submit and block until done. - PreloadSRS/EvictSRS: Explicit control for Curio to warm up SRS ahead of proof arrival, or free memory when switching workload types. - Priority levels: WinningPoSt is CRITICAL (preempts other work), WindowPoSt is HIGH (deadline-sensitive), PoRep/SnapDeals are NORMAL. - Vanilla proof inline: At ~50 MB per PoRep C1, streaming over gRPC on a unix socket is fine (~5ms transfer). No need for shared memory complexity.

The PreloadSRS/EvictSRS methods are particularly interesting. They give Curio explicit control over the SRS memory hierarchy. If Curio knows that a WindowPoSt deadline is approaching in 5 minutes, it can call PreloadSRS to ensure the WindowPoSt parameters are hot before the proof request arrives. This is analogous to vLLM's model loading API, where the orchestrator can pre-load models before routing requests to them.

The assistant also includes fields like request_id (for idempotency), sector_number, miner_id, and partition_index — all derived from the existing Curio task layer. This shows attention to how the daemon will integrate with the existing system, not just how it works internally.

Section 5: The SRS Memory Manager — Tiered Residency

This section is the heart of the architecture. The assistant proposes a three-tier memory hierarchy inspired by vLLM's sleep mode:

| Tier | Storage | Access Time | Description | |---|---|---|---| | Hot | CUDA pinned host RAM | Immediate | Ready for GPU DMA. Used during active proving. | | Warm | mmap'd file (page cache) | ~2-10s to re-pin | File is mmap'd, OS page cache may have it. Re-pinning = cudaHostRegister on mmap region. | | Cold | Disk only | 30-90s | Full deserialization from .params file. |

The Rust struct for the SRS manager shows the implementation:

struct SRSManager {
    pinned_budget: u64,
    hot: HashMap<CircuitId, Arc<SRS>>,
    warm: HashMap<CircuitId, MmapHandle>,
    ref_counts: HashMap<CircuitId, AtomicU32>,
    lru: VecDeque<CircuitId>,
}

The eviction policy is carefully designed:

  1. Never evict an SRS with ref_count > 0 (in-flight proofs)
  2. Evict LRU entry with ref_count == 0 from hot → warm (unpin, keep mmap)
  3. If warm budget exceeded, evict LRU warm → cold (munmap)
  4. Before starting a proof, ensure the required SRS is hot. If not, promote warm→hot or cold→hot. This policy balances several competing concerns: - Safety: In-flight proofs are never interrupted by SRS eviction - Efficiency: LRU eviction minimizes the cost of future promotions - Predictability: Explicit budget prevents OOM The assistant then maps this strategy to specific machine configurations: Small machine (96-128 GiB RAM): - Pinned budget: ~48 GiB (one PoRep SRS + PoSt SRS) - When proving PoRep, PoSt SRS stays hot (only ~2 GiB) - When switching from PoRep to SnapDeals: evict PoRep SRS to warm, load SnapDeals SRS - Cost of swap: ~30-60s (acceptable if infrequent) Large machine (256+ GiB RAM): - Pinned budget: ~100-140 GiB - Keep PoRep + SnapDeals + all PoSt SRS hot simultaneously - Zero swap overhead for any proof type The assistant also proposes "proactive SRS warming" — the scheduler predicts upcoming SRS needs based on the queue and starts loading SRS data before the proof arrives. This is directly inspired by inference engines that pre-load models based on anticipated request patterns.

The Assumption Behind the Memory Manager

The SRS memory manager makes a critical assumption: that CUDA pinned memory can be allocated and freed without significant fragmentation. The assistant assumes that pinning an mmap'd region with cudaHostRegister and unpinning it with cudaHostUnregister is a fast operation (~2-10s for a 47 GiB region). This assumption is based on the existing supraseal code, which uses cudaHostRegister for its SRS cache. However, the assistant does not verify this assumption experimentally — it's derived from reading the code, not from measurement.

There's also an implicit assumption that the OS page cache will retain mmap'd data after unpinning. This is true under normal circumstances, but if the system is under memory pressure, the kernel may evict pages from the page cache, turning a warm→hot promotion into a cold→hot promotion. The assistant's estimated access times (~2-10s for warm, 30-90s for cold) suggest an awareness of this variability, but the design doesn't include mechanisms to prevent or detect page cache eviction.

Section 6: The Scheduler Design — Priority, Batching, and Affinity

The scheduler design is the most architecturally complex component. It has four layers:

  1. Admission Control: Enforces memory budget before accepting a proof
  2. Priority Queues: Three levels (CRITICAL, HIGH, NORMAL) with strict priority ordering
  3. Batch Collector: Accumulates same-circuit-type jobs into batches
  4. GPU Affinity Router: Routes proofs to GPUs that already have the right SRS loaded The scheduling rules are:
  5. CRITICAL preempts everything — WinningPoSt interrupts ongoing PoRep synthesis
  6. Batch accumulation — same-circuit-type jobs accumulate, flushed when batch full, timeout, or high-priority arrival
  7. GPU affinity — prefer GPU with matching SRS loaded
  8. Pipeline overlap — synthesize next batch while GPU computes current batch The assistant also defines two operating modes: Small Machine Mode (1 GPU, 96 GiB): - Batch size: 1 (no batching — insufficient RAM) - Pipeline depth: 2 (synthesize next while GPU proves current) - SRS strategy: one primary type hot, swap on demand - Reorder NORMAL queue to group by circuit type Large Machine Mode (4 GPUs, 512 GiB): - Dedicate GPUs by type: gpu:0,1 for PoRep, gpu:2 for WindowPoSt, gpu:3 floats - Batch size: 3 per GPU for PoRep - Pipeline depth: 3 (synthesis, GPU compute, result assembly overlap) - All SRS hot simultaneously This dual-mode design is a direct response to the user's requirement that the daemon work "both equally" on small and large machines. The assistant doesn't just scale parameters — it changes the scheduling strategy entirely based on available resources.

The Preemption Decision

One of the most carefully considered decisions is about preemption. The assistant states:

Preemption: Queue-level only (no mid-compute). CUDA kernels can't be safely interrupted; GPU phase is bounded (~85s max).

This is a pragmatic choice. True preemption (killing a running CUDA kernel) would require either:

Section 7: The GPU Worker Pipeline — From Sequential to Pipelined

The GPU worker pipeline is described as a loop with seven steps:

  1. ACQUIRE job from scheduler
  2. ENSURE SRS is hot (may block on promotion)
  3. SYNTHESIZE witnesses (CPU thread pool)
  4. SUBMIT to GPU (generate_groth16_proofs_c)
  5. COLLECT proof results
  6. RESPOND to client (gRPC AwaitProof)
  7. RELEASE SRS ref (decrement ref_count) The critical insight is that steps 3 and 4 can be pipelined across sequential jobs: "Job N GPU compute || Job N+1 synthesis." This is the core pipelining win that Phase 2 enables. The assistant distinguishes between Phase 1 (scaffold, no pipelining) and Phase 2+ (pipelined with split synthesis/prove API). In Phase 1, the worker simply calls the existing seal_commit_phase2() function — no upstream modifications needed. In Phase 2, after modifying bellperson to expose separate synthesis and prove functions, the worker can overlap these phases. This phased approach is a key architectural strategy: deliver value immediately with no upstream changes, then unlock more performance as modifications are made. The scaffold is not a throwaway prototype — it's a production-quality system that happens to not use pipelining yet.

Section 8: Library / Binary Design — The Crate Structure

The crate structure is:

extern/cuzk/
├── cuzk-core/          # Core library: Engine, Scheduler, SRS Manager, GPU Worker
├── cuzk-proto/         # Protobuf definitions
├── cuzk-server/        # gRPC server (thin wrapper over core)
├── cuzk-daemon/        # Standalone binary
└── cuzk-ffi/           # C ABI for embedding in Go (Curio)

This separation of concerns is standard Rust workspace design, but the inclusion of cuzk-ffi is notable. It provides a C ABI that Curio can link against using CGO, enabling three deployment modes:

type Client struct {
    conn *grpc.ClientConn
}

func (c *Client) Prove(ctx context.Context, req *cuzkpb.ProveRequest) (*cuzkpb.ProveResponse, error) {
    return c.stub.Prove(ctx, req)
}

This replaces ffiselect.FFISelect.SealCommitPhase2() etc. in the Curio task code. The assistant notes that this is "feature-gated behind config flag" and "falls back to ffiselect if daemon unavailable" — a careful migration strategy that doesn't break existing deployments.

Section 9: The Phased Implementation Roadmap — 18 Weeks to 10x

The roadmap is the most concrete part of the message. It spans 18 weeks across 6 phases, each with specific deliverables, estimated impact, and stopping points.

Phase 0: Scaffold (Weeks 1-3) — "It proves with SRS residency"

The key insight of Phase 0 is that zero upstream modifications are needed to achieve a 25% throughput improvement. The daemon simply:

  1. Pre-populates GROTH_PARAM_MEMORY_CACHE at startup by loading SRS once
  2. Keeps the SRS in memory via Arc reference counting
  3. Calls the existing seal_commit_phase2() function for each proof The SRS residency is achieved through a clever trick: the existing filecoin-proofs crate uses a lazy_static HashMap called GROTH_PARAM_MEMORY_CACHE to cache loaded parameters. By calling the parameter loading function at daemon startup (before any proof requests arrive), the parameters are loaded into this cache and stay there for the daemon's lifetime. Subsequent proof calls hit the cache and skip disk I/O. The assistant estimates this eliminates 30-90s of SRS load per proof, yielding ~25% throughput increase. This is immediate, measurable value with no risk of breaking existing functionality.

Phase 1: Multi-Type Scheduling (Weeks 3-5) — "It handles all proof types smartly"

Phase 1 adds the priority queue scheduler, SRS warm tier, GPU affinity tracking, and support for all proof types. The key deliverable is that WinningPoSt never misses a deadline due to PoRep hogging the GPU.

The SRS swapping strategy is described in detail:

Phase 2: Pipelining (Weeks 5-8) — "GPU never idles"

Phase 2 requires the first upstream modification: splitting bellperson's create_proof_batch_priority_inner into separate synthesis and prove functions. The assistant describes this as a "small, low-risk modification" and provides a concrete plan:

Phase 3: Batching (Weeks 8-11) — "Multiple sectors per GPU pass"

Phase 3 batches multiple sectors' circuits into a single generate_groth16_proofs_c call. This requires:

Phase 4: Compute Optimizations (Weeks 11-14) — "Faster per-proof"

Phase 4 cherry-picks the highest-impact micro-optimizations from the earlier Proposal 4:

  1. SmallVec for LC Indexer: 15-30% synthesis speedup, ~5 LOC
  2. Pre-size vectors: 5-10% synthesis speedup
  3. Pin a,b,c with cudaHostRegister: +50% transfer bandwidth
  4. Parallelize B_G2 MSMs: 50s → 5s, one-line change
  5. batch_addition occupancy: 5-12% MSM speedup, one-line change Estimated impact: 30-40% faster per-proof on top of batching gains.

Phase 5: PCE — Pre-Compiled Constraint Evaluator (Weeks 14-18) — "Skip synthesis"

Phase 5 is the most ambitious: eliminating circuit synthesis overhead by pre-compiling R1CS constraint matrices. This is described as "the biggest single optimization" with 3-5x synthesis speedup.

The deliverables include:

  1. RecordingCS to extract fixed R1CS matrices into CSR format
  2. WitnessCS-based witness generation (skip enforce(), only run alloc())
  3. Sparse MatVec evaluator for a = A·w, b = B·w, c = C·w
  4. Coefficient-specialized MatVec (±1 skip multiply, boolean witness fast-path)
  5. Pre-sorted SRS topology for split MSM

The Cumulative Impact Table

The message includes a stopping-points table showing cumulative throughput at each phase:

| After Phase | Throughput vs Baseline | Minimum RAM | Key Win | |---|---|---|---| | 0 | 1.3x | 256 GiB (no change) | SRS residency | | 1 | 1.3x (scheduling) | 256 GiB | Mixed workload correctness | | 2 | 1.8x | 128 GiB | GPU pipelining | | 3 | 4-5x | 128 GiB | Batching | | 4 | 6-7x | 96 GiB | Compute wins | | 5 | 10x+ | 96 GiB | PCE |

The decreasing minimum RAM requirement is notable: as optimizations reduce per-proof memory, the daemon becomes viable on smaller machines. This is a deliberate design goal, responding to the user's requirement for both small and large machine support.

Section 10: Configuration — The TOML Schema

The configuration file schema shows the assistant's attention to operational details:

[daemon]
listen = "unix:///run/curio/cuzk.sock"

[memory]
pinned_budget = "50GiB"
working_memory_budget = "80GiB"

[gpus]
devices = []
# affinity: 0 = "porep-32g", 1 = "any"

[scheduler]
max_batch_size = 3
max_batch_wait_ms = 10000
sort_by_type = true

[synthesis]
threads = 0  # auto

[srs]
param_cache = "/var/tmp/filecoin-proof-parameters"
preload = ["porep-32g"]

[logging]
level = "info"

Every configuration parameter has a rationale. The pinned_budget of 50 GiB on a 96 GiB machine leaves 46 GiB for synthesis working memory and OS overhead. The max_batch_wait_ms of 10000 prevents proofs from waiting too long for a batch to fill. The sort_by_type option reduces SRS swaps on small machines. The preload array lets operators decide which SRS files to warm at startup.

The assistant also includes commented-out examples for GPU affinity, showing how to pin specific circuit types to specific GPUs. This is a power-user feature that requires understanding of the workload patterns.

Section 11: Test Harness — Golden Data and Test Modes

The test harness design shows the assistant thinking about testing strategy:

extern/cuzk/testdata/
├── porep-32g-c1.json
├── porep-32g-proof.json
├── snap-32g-vanilla.json
├── wpost-32g-vanilla.json
└── winning-32g-vanilla.json

Three test modes are proposed:

  1. Integration test: Load real SRS, prove with golden input, verify proof against expected output
  2. Mock test: Mock the supraseal FFI layer, test scheduling/SRS management/gRPC without GPU
  3. Benchmark: Repeated proofs with timing, memory tracking, GPU utilization The mock test mode is particularly important for development. It allows testing the scheduler, SRS manager, and gRPC API without requiring a GPU — enabling rapid iteration and CI/CD integration. The assistant is thinking about the development workflow, not just the final product.

Section 12: Key Design Decisions & Tradeoffs

The decision matrix summarizes the major architectural choices:

| Decision | Choice | Rationale | |---|---|---| | Language | Rust (tokio) | Direct access to bellperson/supraseal, no CGO overhead, better memory control | | RPC | gRPC (tonic) | Strongly typed, streaming, mature Rust+Go ecosystem | | SRS management | Tiered (hot/warm/cold) | Inference engine pattern, works for both small and large machines | | Batching granularity | Per-circuit-type | All circuits in a batch must share the same R1CS structure and SRS | | Preemption | Queue-level only | CUDA kernels can't be safely interrupted; GPU phase is bounded | | Library vs binary | Library with exec mode | Maximum flexibility: embed, standalone, or child process | | Upstream modifications | Phase 0: zero; Phase 2+: fork | Scaffold is immediately useful; deeper optimizations require controlled fork | | Error handling | Retry at daemon level | Daemon detects GPU OOM/fault, retries on different GPU or after SRS eviction |

Each decision is justified with a concise rationale. The assistant is not making arbitrary choices — each decision is traced back to a constraint or requirement discovered during the investigation phase.

Section 13: Open Questions — Honest Uncertainty

The message ends with four open questions:

  1. SnapDeals partition count: SnapDeals has 16 partitions vs PoRep's 10. Does the current supraseal code handle num_circuits &gt; 10? Need to verify max_num_circuits handling.
  2. Non-supraseal path for PoSt: In default Curio builds, PoSt uses bellperson's native prover (not supraseal). Should cuzk support both prover backends? Or require cuda-supraseal for all proof types?
  3. SnarkPack aggregation: Should cuzk also handle proof aggregation (CPU-only, no GPU)? Or leave that to Curio?
  4. Remote proving: The gRPC API supports TCP listeners for remote proving (proofshare marketplace). Should this be in scope for Phase 0, or deferred? These open questions are not weaknesses — they are evidence of intellectual honesty. The assistant knows the limits of its investigation and explicitly marks areas that need further exploration. This is a hallmark of good engineering: knowing what you don't know.

Assumptions Embedded in the Design

Let us now examine the assumptions that underpin this architecture. Some are explicit, some are implicit, and some may be incorrect.

Explicit Assumptions

  1. C1 output is ~50 MB, fits over gRPC: Verified by checking the actual file (message 72). Correct.
  2. SRS loading takes 30-90s: Based on the prior investigation of the existing pipeline. Reasonable but not experimentally verified for all proof types.
  3. CUDA pinned memory can be allocated/freed without fragmentation: Based on reading the supraseal code. Needs experimental validation.
  4. GPU phase is bounded at ~85s max: Based on the prior analysis of NTT/MSM timing. Reasonable for current hardware.
  5. The inference engine analogy is structurally valid: This is the core architectural insight. It is well-supported by the mapping table.

Implicit Assumptions

  1. The OS page cache will retain mmap'd SRS data after unpinning: This is true under normal conditions but can fail under memory pressure. The design doesn't include mechanisms to detect or recover from page cache eviction.
  2. The scheduler can predict SRS needs from the queue: This assumes that proof requests arrive with enough lead time to pre-load SRS. For burst workloads (many proofs arriving simultaneously), this may not hold.
  3. WinningPoSt can tolerate up to ~85s of waiting: The Filecoin epoch is 30 seconds. If a WinningPoSt proof arrives during the GPU phase of a PoRep proof, it will miss the deadline. The assistant assumes this is rare enough to be acceptable, but doesn't quantify the risk.
  4. The bellperson modification for Phase 2 is "small, low-risk": This is an engineering judgment that may prove optimistic when actually implementing.
  5. All proof types can use the supraseal CUDA backend: The assistant notes that PoSt currently uses bellperson's native prover in default builds. The design assumes this can be changed.

Potentially Incorrect Assumptions

  1. The SnapDeals SRS size is ~30 GiB (estimated): Marked with an asterisk, this is acknowledged as uncertain. If the actual size is significantly different, the memory budget calculations may be off.
  2. max_num_circuits can be bumped from 10 to 30+: This depends on GPU memory capacity and the CUDA kernel's internal assumptions. The assistant hasn't verified this experimentally.
  3. The tiered memory hierarchy access times: The estimated times (immediate for hot, ~2-10s for warm, 30-90s for cold) are plausible but not measured. The warm→hot promotion time depends on cudaHostRegister performance, which may vary by GPU and driver version.
  4. The scheduler can reorder NORMAL-priority proofs without side effects: Reordering proofs changes the order in which sectors are sealed, which could affect Curio's task tracking and deadline calculations.

Knowledge Inputs and Outputs

Input Knowledge Required to Understand This Message

To fully understand message 77, one needs:

  1. Groth16 proof generation: Understanding of R1CS, witness synthesis, NTT, MSM, and the structure of a SNARK proof
  2. Filecoin proof types: PoRep (Proof of Replication), SnapDeals, WindowPoSt, WinningPoSt — what each proves and when it's needed
  3. CUDA memory model: Pinned host memory, device memory, DMA transfers, kernel launches
  4. GPU inference engine architecture: vLLM, Triton, TensorRT-LLM — model loading, scheduling, batching
  5. Rust async programming: tokio, tonic (gRPC), async task management
  6. Curio architecture: The ffiselect child process model, task layer, proof sharing
  7. The prior optimization proposals: The nine bottlenecks, 18 micro-optimizations, PCE design

Output Knowledge Created by This Message

This message creates:

  1. A complete architecture for a SNARK proving daemon: The first documented design for a persistent, GPU-resident proof generation system in the Filecoin ecosystem
  2. A phased implementation roadmap: 18 weeks, 6 phases, with clear deliverables and stopping points
  3. A resource requirement matrix: For all proof types, enabling capacity planning
  4. A gRPC API specification: The protobuf schema that defines the interface between Curio and the daemon
  5. A memory management strategy: The tiered hot/warm/cold hierarchy with eviction policies
  6. A scheduler design: Priority queues, batch accumulation, GPU affinity, deadline awareness
  7. A deployment model: Three modes (exec, spawn, external) with the same protocol
  8. A test strategy: Golden data, integration/mock/benchmark modes
  9. A configuration schema: TOML-based with sensible defaults
  10. A set of open questions: Areas requiring further investigation

The Thinking Process: From Investigation to Design

The most remarkable aspect of message 77 is not the architecture itself — it's the thinking process that produced it. Let us trace this process through the conversation.

Step 1: Understanding the Problem Space (Messages 68-76)

The assistant began by reading all seven prior proposals and background documents. This gave it a comprehensive understanding of the existing system: the call chain, the memory footprint, the bottlenecks, and the optimization opportunities.

Then it systematically explored every layer of the system:

Step 2: Finding the Right Analogy (Message 68)

The assistant then studied GPU inference engine architectures. This was a deliberate search for patterns — the assistant was looking for existing solutions to the resource management problem it had identified. The inference engine analogy provided:

Step 3: Verifying Key Facts (Messages 72-75)

Before designing, the assistant verified critical assumptions:

Step 4: Synthesizing the Architecture (Message 77)

With all the investigation complete, the assistant synthesized the architecture. The synthesis process involved:

  1. Identifying the core value proposition: SRS residency (eliminate per-proof loading)
  2. Finding the minimal viable system: Phase 0 with zero upstream modifications
  3. Layering on complexity: Each phase adds one new capability
  4. Drawing from the analogy: The inference engine patterns provided templates for the scheduler, memory manager, and worker pipeline
  5. Accounting for constraints: Small vs large machines, different proof types, priority levels
  6. Acknowledging uncertainty: Open questions and estimated values

Step 5: Writing the Document

The final message is not just a design — it's a communication document. It's structured to be read by different audiences:

Mistakes and Potential Weaknesses

No design is perfect. Let us examine potential weaknesses in the cuzk architecture.

Weakness 1: The Preemption Gap

The most significant potential weakness is the preemption strategy. WinningPoSt proofs must complete within a Filecoin epoch (30 seconds). If a WinningPoSt arrives during the GPU phase of a PoRep proof (~85s), it will miss the deadline. The assistant's mitigation is to preempt at the synthesis stage (before GPU submission), but this doesn't cover the case where the GPU is already busy.

A more robust approach might be:

Weakness 2: The Warm Tier Assumption

The warm tier assumes that mmap'd SRS data remains in the OS page cache after unpinning. This is a reasonable assumption under normal conditions, but it's not guaranteed. Under memory pressure, the kernel may evict pages, turning a warm→hot promotion into a cold→hot promotion (30-90s instead of 2-10s).

The design doesn't include mechanisms to:

Weakness 3: Batch Size Limits

The assistant assumes that max_num_circuits can be bumped from 10 to 30+. This is based on reading the CUDA kernel header file, but the actual limit depends on GPU memory capacity and the kernel's internal data structures. If the limit is hard-coded in multiple places (not just the header), changing it may require more extensive modifications.

Additionally, the assistant doesn't consider the interaction between batch size and GPU memory fragmentation. Larger batches require more GPU memory, which may increase fragmentation and reduce the effective batch size over time.

Weakness 4: The Bellperson Modification Risk

Phase 2 requires modifying bellperson to split synthesis and proving into separate functions. The assistant describes this as "small, low-risk," but any modification to a cryptographic library carries risk:

Weakness 5: Single-Point-of-Failure Architecture

The daemon is a single process that manages all GPUs, all SRS memory, and all proof scheduling. If it crashes, all in-flight proofs are lost. The design doesn't include:

The Broader Significance: A New Category of Infrastructure

Message 77 is significant beyond its immediate context. It represents the design of a new category of infrastructure: a SNARK proving engine that treats proof generation as a service rather than a function call.

The existing architecture calls seal_commit_phase2() as a blocking function — load SRS, synthesize witnesses, compute proof, return result, free everything. Each call is independent, stateless, and starts from scratch. The cuzk architecture transforms this into a persistent service with stateful resource management, request scheduling, and pipeline parallelism.

This shift is analogous to the evolution of machine learning inference:

Conclusion: The Art of Systems Architecture

Message 77 is a masterclass in systems architecture. It demonstrates:

  1. Deep domain understanding: The assistant spent hours reading source code, tracing call chains, and understanding memory layouts. The architecture is grounded in this understanding.
  2. Cross-domain pattern transfer: The inference engine analogy is not superficial — it's structurally precise and drives concrete design decisions.
  3. Pragmatic incrementalism: Phase 0 delivers immediate value with zero risk. Each subsequent phase adds complexity only when the foundation is stable.
  4. Honest uncertainty: The open questions, estimated values, and asterisks show that the assistant knows the limits of its knowledge.
  5. Communication design: The document is structured for multiple audiences, with tables, diagrams, and code snippets that convey information efficiently.
  6. Tradeoff awareness: Every major decision is accompanied by a rationale and, in many cases, an alternative considered and rejected. The message is not the end of the conversation — it's the beginning of implementation. But it's a crucial milestone: the moment when understanding crystallizes into a plan. The cuzk architecture, as designed in message 77, represents a new way of thinking about SNARK proof generation — not as a function call, but as a service. Whether the implementation succeeds or fails, the architecture document stands as a model of how to design complex systems: investigate thoroughly, find the right analogies, design incrementally, and be honest about uncertainty. That is the lesson of message 77.