The Integration Point: Wiring Cross-Sector Batching into the cuzk Proving Engine

The Message

[assistant] Now I need to rewrite the synthesis task in engine.rs to use the batch collector. This is a major change. Let me replace the pipeline mode synthesis task: [edit] /home/theuser/curio/extern/cuzk/cuzk-core/src/engine.rs Edit applied successfully.

At first glance, this message from the cuzk proving engine development session appears almost trivial — a brief announcement followed by a file edit confirmation. But this brevity belies the message's true significance. It represents the architectural keystone of Phase 3 (cross-sector batching) for the cuzk pipelined SNARK proving engine, the moment when all the independently constructed components — the batch collector, the multi-sector synthesis function, the proof splitter, and the GPU worker channel — were finally wired together into a coherent whole. Understanding why this message was written, and what it accomplishes, requires tracing the threads of a complex, multi-phase engineering effort spanning hundreds of lines of Rust code across half a dozen modules.

The Context: Building Phase 3 Piece by Piece

To grasp the weight of this message, one must understand what preceded it. The cuzk project is a persistent GPU-resident SNARK proving engine for Filecoin, designed to replace the existing architecture where each proof spawns a fresh child process that loads a ~47 GiB SRS (Structured Reference String), runs one proof, and exits. Phase 1 established the basic daemon with all four Filecoin proof types (WinningPoSt, WindowPoSt, PoRep C2, SnapDeals) and a multi-GPU worker pool. Phase 2 introduced the pipelined architecture where CPU-bound circuit synthesis for proof N+1 runs concurrently with GPU-bound proving for proof N, achieving a 1.27x throughput improvement over sequential execution.

Phase 3 — cross-sector batching — aimed to go further. The core insight is that multiple sectors' proof circuits share the same R1CS constraint structure and the same SRS. Instead of proving each sector's 10 partitions independently, the engine could accumulate same-type proof requests, synthesize all N×10 partition circuits together in a single combined pass, and prove them in one GPU invocation. This amortizes fixed costs (CUDA kernel compilation, MSM setup, memory allocation) across multiple proofs and improves GPU SM utilization during the batch_addition phase.

The assistant had been building toward this integration across several preceding messages. In message 668, it created batch_collector.rs — a new module implementing a BatchCollector that accumulates same-circuit-type proof requests and flushes them when max_batch_size is reached or max_batch_wait_ms expires. In message 670, it added synthesize_porep_c2_multi() to pipeline.rs, the core function that takes N sectors' C1 outputs and produces a single combined SynthesizedProof. In message 671, it added a split_batched_proofs() test. In message 672, it updated lib.rs to export the new module. And in message 674, it modified the SynthesizedJob type in engine.rs to carry batch metadata — specifically, the information needed to split the combined GPU proof output back into per-sector results and notify the correct individual callers.

Each of these pieces was independently correct and testable. But they were disconnected. The batch collector sat idle. The multi-sector synthesis function had no caller. The GPU worker had no way to receive a batched job. The engine's synthesis task — the central loop that pulls proof requests from the scheduler, dispatches synthesis work, and sends results to the GPU channel — was still the Phase 2 version, handling one proof at a time. Message 675 is the moment when all these pieces were joined.

Why This Was a "Major Change"

The assistant's own characterization — "This is a major change" — is not hyperbole. The synthesis task in engine.rs is the heart of the pipelined proving engine. Its rewrite touched the fundamental control flow of the entire daemon. Before the change, the synthesis task looped as follows: pull a single ProofRequest from the scheduler, call the appropriate synthesis function (e.g., synthesize_porep_c2()), wrap the result in a SynthesizedJob, and send it to the GPU worker channel. The GPU worker would prove it, and the result would be returned to the single caller.

After the change, the synthesis task needed to handle two distinct flows:

  1. Batchable proof types (PoRep C2, SnapDeals): Instead of synthesizing immediately, the task submits the request to the BatchCollector. The batch collector accumulates requests of the same circuit type. When the batch reaches max_batch_size or the max_batch_wait_ms timer expires, the collector flushes: it calls synthesize_porep_c2_multi() (or the SnapDeals equivalent) with all accumulated sectors' C1 outputs, producing a single SynthesizedProof containing all N×10 partition circuits. This is wrapped in a SynthesizedJob that carries the batch metadata — crucially, a mapping from each original caller's channel to their position in the batch. The GPU worker receives this single job, proves all circuits in one invocation, and then split_batched_proofs() separates the concatenated proof bytes back into per-sector results. Each caller receives their individual proof with accurate timing information.
  2. Non-batchable proof types (WinningPoSt, WindowPoSt): These priority-critical proofs cannot be delayed by batching. When one arrives, the synthesis task must preempt-flush any pending batch (sending whatever has accumulated so far to the GPU), then immediately synthesize and prove the priority proof. This ensures that WinningPoSt — which must complete within a Filecoin epoch (~30 seconds) — never waits for a batch timer. This dual-flow design required fundamental restructuring of the synthesis task's state machine. The task now needed to hold a reference to the BatchCollector, check proof types before dispatching, handle the preempt-flush logic, and manage the more complex result-routing path where a single GPU job produces multiple callbacks.

The Decisions Embedded in This Message

While the message itself does not enumerate decisions, the edit it describes encodes several architectural choices:

Decision 1: Batch collector as a synthesis-layer component. The batch collector could have been placed at the scheduler level (accumulating requests before they reach the synthesis task) or at the gRPC service level (accumulating before they enter the engine at all). Placing it at the synthesis layer means the scheduler retains its priority-ordering role — high-priority proofs can still jump the queue — while batching happens only for same-type, same-priority requests that have passed through scheduling. This preserves the scheduler's ability to reorder by priority while still enabling batching for the common case.

Decision 2: Preempt-flush, not preempt-cancel. When a priority proof arrives, the design flushes the current batch rather than canceling it. This means partial batches are still useful — even a batch of size 1 is processed normally. The alternative (discarding the partial batch and restarting after the priority proof) would waste the synthesis work already done. The flush approach ensures no work is lost.

Decision 3: Batch metadata in SynthesizedJob. The SynthesizedJob type was extended in message 674 to carry the information needed to route results back to individual callers. This is a design choice that keeps the GPU worker generic — it doesn't need to know about batching at all. It receives a SynthesizedJob, calls prove_from_assignments(), and returns the combined proof bytes. The result-routing logic lives in the synthesis task and the engine's result-handling code, not in the GPU worker.

Decision 4: Backward compatibility via configuration. The batch collector's max_batch_size parameter defaults to 1, which preserves Phase 2 behavior exactly — each proof is synthesized and proved individually. Setting max_batch_size = 2 or 3 enables batching without any code changes elsewhere. This configuration already existed in config.rs from earlier phases, demonstrating forward-thinking design.

Input Knowledge Required

To understand what message 675 accomplishes, one must already understand:

Output Knowledge Created

Message 675 produced a working integration of cross-sector batching into the cuzk proving engine. Specifically, it created:

Assumptions and Potential Risks

The rewrite makes several assumptions that warrant scrutiny:

Assumption 1: The batch collector's flush is atomic and race-free. The synthesis task and the batch collector run in a tokio async context. If a flush is triggered simultaneously by a batch-size threshold and a timeout, the collector must handle this correctly without double-flushing or losing requests. The Rust implementation uses a Mutex-protected inner state, but the correctness depends on the flush logic being idempotent.

Assumption 2: Preempt-flush is safe for partial batches. When a priority proof arrives, the synthesis task flushes whatever is in the batch collector, even if the batch is only partially full. This means a batch of size 1 (a single PoRep request) will be synthesized and proved immediately, which is correct but may not be optimal. The assumption is that this is always better than discarding the partial batch.

Assumption 3: Non-batchable types are rare enough that preempt-flush overhead is negligible. WinningPoSt proofs are infrequent (once per epoch, ~30 seconds) and WindowPoSt proofs are also relatively rare. The overhead of flushing a partial batch when one arrives is small compared to the total proof time (~90 seconds for PoRep). But if the workload mix changes — for example, if WindowPoSt proofs become more frequent — the constant preempt-flushing could reduce batching efficiency.

Assumption 4: The GPU worker can handle arbitrarily large batches. The max_num_circuits constant in the supraseal C++ code was 10 in the original codebase. For cross-sector batching with 3 sectors of 10 partitions each, this needs to be 30. The assistant noted in earlier messages that this constant is "only used for thread heuristics" and can be bumped. But there may be GPU memory limits or kernel launch parameters that constrain batch size in practice.

The Broader Significance

Message 675 is a testament to a particular engineering philosophy: build the components independently, test them in isolation, and then integrate them in a single focused effort. The batch collector, multi-sector synthesis, proof splitting, and engine modifications were each developed and verified separately. The integration in message 675 was the culmination — the moment when the architecture shifted from "Phase 2 with some new files" to "Phase 3, fully operational."

The subsequent GPU E2E validation (documented in the chunk summary for this segment) confirmed the design's correctness. On an RTX 5070 Ti with real 32 GiB PoRep data, batch_size=2 achieved 121.6 seconds for two proofs (60.8s per proof steady-state), compared to 177.8 seconds for two sequential single proofs — a 1.46x throughput improvement. Peak RSS memory was only ~7.5 GiB, just ~2 GiB above the single-proof baseline of ~5.5 GiB, because the 47 GiB SRS is shared and only the compressed auxiliary assignments grow linearly with batch size.

This message, for all its brevity, represents the architectural keystone of Phase 3. It is the point where design becomes reality, where isolated components become a system, and where the promise of cross-sector batching — better GPU utilization, amortized fixed costs, higher throughput — becomes measurable in real hardware.