The Fork That Unlocks Pipelined Proving: Bellperson at the Crossroads

In the span of a single bash command, the entire trajectory of the cuzk proving daemon shifted from analysis to implementation. Message 410 in this opencode session is deceptively simple — a cp -r invocation that copies the bellperson Groth16 library from the Rust cargo registry cache into the project's extern/ directory. Yet this copy operation represents the culmination of dozens of prior messages, hundreds of lines of analysis, and a carefully reasoned architectural decision that will determine how the cuzk project achieves its core goal: a pipelined, memory-efficient SNARK proving daemon for Filecoin storage proofs.

The Message

The assistant executed:

cp -r ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bellperson-0.26.0 /home/theuser/curio/extern/bellperson && ls /home/theuser/curio/extern/bellperson/

The output confirmed the copied directory structure:

build.rs
Cargo.toml
Cargo.toml.orig
COPYRIGHT
LICENSE-APACHE
LICENSE-MIT
README_AMD.md
README.md
rust-toolchain
SECURITY.md
src
tests

This is a verbatim copy of bellperson version 0.26.0 from the local cargo registry cache. Every file — from the build script to the license files to the source directory — has been duplicated into the project's workspace. The && ls ensures immediate verification that the copy succeeded and all expected files are present.

The Long Road to This Command

To understand why this simple copy is so significant, one must trace the reasoning that led to it. The cuzk project, now in its seventh segment of development, had just completed Phase 1 — implementing all four Filecoin proof types (WinningPoSt, WindowPoSt, SnapDeals, and the original PoRep seal) and validating them against golden test data with real GPU proofs. Phase 1 demonstrated a 20.5% speedup from SRS (Structured Reference String) residency alone, proving the value of a persistent proving daemon.

Phase 2, however, aims for a far more ambitious transformation: splitting the monolithic Groth16 proof generation into two distinct phases that can be pipelined independently. The CPU-intensive synthesis phase (where circuits are constructed and constraint systems are evaluated) would run on a thread pool, producing intermediate state. The GPU-intensive proving phase (where NTT, MSM, and proof assembly occur) would consume that state on CUDA workers. This split would reduce peak memory from ~200 GiB to ~13.6 GiB per partition and enable continuous, overlapping proof generation.

The critical insight, discovered through painstaking code analysis in messages 395 through 399, is that bellperson's supraseal prover already has this two-phase architecture internally. The function synthesize_circuits_batch() (~60 lines) performs all CPU work: it calls circuit.synthesize() for each circuit in parallel via rayon, producing ProvingAssignment<Scalar> objects containing the a, b, and c evaluation vectors along with density trackers. Immediately after synthesis, the GPU phase (~70 lines) packs raw pointers and calls into supraseal_c2::generate_groth16_proof(), which handles NTT, MSM, and proof assembly on CUDA.

The boundary between these phases is clean and well-defined. The problem is that both functions are private. ProvingAssignment<Scalar> is a crate-private struct. synthesize_circuits_batch() is an internal helper. The GPU-phase code is inline within create_proof_batch_priority_inner(). To expose this split, bellperson must be forked.

Why a Fork, Not a Patch

The decision to create a local fork rather than submit an upstream PR or use a different approach was not made lightly. The assistant explored multiple strategies across messages 398 and 399:

Strategy A: Fork only bellperson — Expose ProvingAssignment as public, make synthesize_circuits_batch() public, and add a new prove_from_assignments() function that encapsulates the GPU phase. This requires ~30 lines of changes, no logic modifications, and zero behavioral changes to existing code paths.

Strategy B: Fork bellperson + storage-proofs-core + filecoin-proofs + filecoin-proofs-api — A cascade of four forks to expose the split at every layer of the call chain. This would provide maximum control but at enormous maintenance cost.

Strategy C: Bypass filecoin-proofs entirely — Replicate the glue code in cuzk-core, calling CompoundProof::circuit() directly (which is public) and then using the split bellperson API. This requires only the bellperson fork plus ~100 lines of glue code in cuzk-core.

The assistant correctly chose Strategy A/C hybrid: fork only bellperson, then add thin glue in cuzk-core. This was validated by the discovery that CompoundProof::circuit() is a public trait method, and all the compound proof types (StackedCompound, FallbackPoStCompound, EmptySectorUpdateCompound) are public. The SRS parameters can be loaded via SuprasealParameters::new(path), which is also public. The only missing piece is bellperson's internal split API.

Assumptions Embedded in the Copy

The copy command itself carries several assumptions, most of which are well-founded but some of which remain unvalidated:

The cargo registry source is authoritative. The assistant assumes that ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bellperson-0.26.0/ contains the correct, complete upstream source. This is the canonical location where cargo stores downloaded crate sources, so the assumption is reasonable — but it depends on the registry having been properly populated during a prior cargo build.

The fork will compile cleanly. The directory includes build.rs, Cargo.toml, rust-toolchain, and all source files. The assumption is that copying these verbatim preserves the build system. However, the fork's location within the workspace — under extern/bellperson/ rather than at a path matching the original crate name — may require Cargo.toml adjustments or workspace-level [patch] directives. The assistant has anticipated this and will later add [patch.crates-io] entries to redirect the bellperson dependency to the local fork.

No transitive dependencies need modification. The assumption is that bellperson's dependencies (bellpepper-core, ec-gpu-gen, ff, etc.) remain unchanged and compatible. Since the fork makes no logic changes — only visibility changes — this is a safe assumption.

The extern/ directory convention is consistent. Previous external forks (supra_seal, supraseal, filecoin-ffi, and the cuzk workspace itself) all live under extern/. Placing bellperson there maintains consistency with the project's established structure.

Input Knowledge Required

To understand the significance of this message, one needs:

• Knowledge of Groth16 proof generation: The two-phase structure (synthesis + proving) is fundamental to how zk-SNARKs work, but in most implementations the phases are tightly coupled. Understanding that splitting them enables pipelining is key.
• Knowledge of the Filecoin proof pipeline: The C2 phase of Filecoin's PoRep (Proof-of-Replication) generates Groth16 proofs across 10 partitions, each containing ~13 million constraints. The memory footprint (~200 GiB peak) is the primary bottleneck this fork aims to address.
• Knowledge of bellperson's architecture: The supraseal prover in bellperson-0.26.0 contains the private synthesize_circuits_batch() function and the inline GPU-phase code that this fork will expose.
• Knowledge of Rust's visibility system: The distinction between pub, crate-private, and private fields determines what can be accessed from outside the crate. The fork's changes are purely about visibility — no logic is modified.
• Knowledge of Cargo's [patch] mechanism: The workspace will use [patch.crates-io] to redirect the bellperson dependency to the local fork, which is how the modified crate gets used without changing every upstream dependency declaration.

Output Knowledge Created

This message produces a concrete artifact: a complete, unmodified copy of bellperson-0.26.0 at /home/theuser/curio/extern/bellperson/. This fork is now ready for modification. The output also includes:

• Verification of the copy: The ls output confirms all 12 files/directories are present, including src/ (the source code), tests/ (the test suite), build.rs (a build script, likely for C++ FFI), and the Cargo.toml manifest.
• A foundation for Phase 2 implementation: With the fork in place, the next steps are clear: make ProvingAssignment public, expose synthesize_circuits_batch(), add prove_from_assignments(), patch the workspace, and validate compilation. The copy is the prerequisite for all of these.
• A commitment to the fork strategy: Before this message, the fork was hypothetical — discussed in design documents and analysis messages. After this message, it is real. The project has crossed a threshold from planning to doing.

The Thinking Process Visible

The assistant's reasoning, visible across the preceding messages, reveals a methodical approach to engineering decision-making:

1. Exhaustive analysis before action: Messages 395-399 read every relevant file in bellperson, storage-proofs-core, filecoin-proofs, and filecoin-proofs-api before making any changes. The assistant traced the full call chain from seal_commit_phase2() down to supraseal_c2::generate_groth16_proof(), verifying every intermediate step.
2. Minimal intervention philosophy: The fork will change approximately 30 lines — all visibility modifiers, no logic. This is a deliberate choice to minimize divergence from upstream and reduce maintenance burden. The assistant explicitly rejected the temptation to rewrite or restructure bellperson's internals.
3. Memory-aware design: The design document (cuzk-phase2-design.md, written in messages 403-406) includes detailed memory budget analysis. The assistant verified constraint counts per partition (~13 million), calculated intermediate state sizes (~136 GiB for all 10 partitions vs ~13.6 GiB per partition), and designed the pipeline queue to hold at most one pre-synthesized PoRep proof on a 256 GiB machine.
4. Verification at every step: The && ls in the copy command is characteristic of the assistant's approach — always verify, never assume. Earlier messages show the same pattern: reading file contents after every edit, running tests after every change, checking directory listings after every copy.
5. Contextual awareness: The assistant knows the project's convention of placing forks under extern/, knows that [patch.crates-io] is the standard Cargo mechanism for local dependency overrides, and knows that the workspace already has other external dependencies with similar structures.

Broader Significance

This message represents the transition from Phase 1 to Phase 2 of the cuzk project. Phase 1 proved that a persistent proving daemon could eliminate SRS loading overhead and achieve a 20.5% speedup. Phase 2 aims to fundamentally restructure how proofs are generated — not just caching parameters, but splitting the computation itself into independently schedulable phases.

The bellperson fork is the keystone of this effort. Without it, the pipelined prover cannot exist — the internal APIs that separate synthesis from GPU proving would remain locked behind Rust's visibility barriers. With it, cuzk can orchestrate a continuous pipeline where CPU cores synthesize circuits for one proof while GPU cores prove another, overlapping computation and dramatically reducing peak memory.

The copy command in message 410 is, in essence, the moment when the cuzk project stopped designing the future and started building it.