The Commit That Unlocks the Pipeline: Forking Bellperson for the cuzk Proving Engine

In the middle of a sprawling coding session dedicated to building a pipelined SNARK proving daemon called "cuzk," there arrives a message that at first glance appears almost trivial. Message [msg 427] contains just two lines from the assistant: "Good, no target directory. Let me commit:" followed by a git add and git diff --cached --stat command showing the files being staged. The output is a list of file paths and line counts — a design document at 791 lines, a bellperson fork with its Cargo.toml, license files, CI configuration, and more.

But this message is anything but trivial. It is the culmination of a carefully orchestrated sequence of analysis, design, and surgical code modification that spanned dozens of preceding messages. It represents the moment when the theoretical architecture for a pipelined proof generation system crossed into concrete, committed code. To understand why this message matters, one must trace the chain of reasoning that led to it — a chain that reveals deep assumptions about software architecture, memory budgets, and the art of minimal forking.

The Strategic Context: Why Phase 2 Exists

The cuzk project (short for "Curio SNARK proving daemon") was conceived as a solution to a specific, painful problem in the Filecoin proof generation pipeline. The existing system, built around the supraseal-c2 library, generates Groth16 proofs for storage proofs (PoRep, PoSt, SnapDeals) in a monolithic fashion: each proof request loads the entire Structured Reference String (SRS), performs circuit synthesis on the CPU, then runs the GPU-accelerated prover. The SRS loading alone costs roughly 20% of wall-clock time, and the peak memory footprint hovers around 200 GiB for a single PoRep proof. In a cloud rental market where machines are billed by the hour, this architecture wastes both time and money.

The earlier phases of the cuzk project (Phase 0 and Phase 1) had already built the scaffolding: a gRPC-based daemon with a priority scheduler, multi-GPU worker pool with CUDA_VISIBLE_DEVICES isolation, support for all four Filecoin proof types (WinningPoSt, WindowPoSt, SnapDeals, and PoRep), and a benchmarking tool. But the actual proof generation still called through the existing monolithic pipeline. The promised speedups — SRS residency saving 20.5%, per-partition streaming reducing peak memory from ~200 GiB to ~13.6 GiB — required a fundamental architectural change: splitting the proof generation into two decoupled phases that could run on separate machines, with synthesis on cheap CPU instances and the GPU phase on expensive GPU instances.

Phase 2 was the plan to implement this split. And the key enabler was a fork of the bellperson library.

The Critical Insight: The Split Already Exists

The assistant's journey toward this commit began with a deep analysis of bellperson's internals in [msg 400]. The critical discovery was that the synthesis/GPU split already existed inside bellperson's codebase, just not exposed publicly. The function synthesize_circuits_batch() already performed the CPU-intensive constraint synthesis, and the GPU-phase code lived in create_proof_batch_priority_inner(). These were internal implementation details, hidden behind pub(super) and pub(crate) visibility modifiers.

This discovery shaped the entire approach. The assistant did not need to write a pipelined prover from scratch — it needed to expose the one that already existed. This is a fundamentally different engineering philosophy: instead of rewriting, refactor the visibility boundaries. The minimal-fork approach meant copying the upstream bellperson source (version 0.26.0) into the project's extern/ directory and making approximately 130 lines of changes across four files:

  1. Cargo.toml: Change the package name and version to avoid conflicting with the upstream crate.
  2. prover/mod.rs: Make the ProvingAssignment struct and its fields pub, make the supraseal module pub, and make synthesize_circuits_batch() pub.
  3. prover/supraseal.rs: Add a new public function prove_from_assignments() that extracts the GPU-phase logic from the existing create_proof_batch_priority_inner().
  4. groth16/mod.rs: Re-export the newly public types so external code can reach them. The elegance of this approach is that it changes nothing about how the existing code works. The create_proof_batch_priority_inner function remains pub(super) — visible only within the prover module, exactly as before. The new prove_from_assignments is a thin wrapper that calls the same internal logic. The existing tests (all 8 of them) continue to pass without modification. The fork is a pure API surface extension, not a behavioral change.

The Design Document: 791 Lines of Memory-Aware Architecture

Before touching a single line of code, the assistant wrote a 791-line design document at cuzk-phase2-design.md ([msg 403]). This document was not a luxury — it was a necessity born from the complexity of the memory problem. The intermediate state produced by circuit synthesis — the ProvingAssignment structures containing the A, B, C constraint matrices and the H, U, L, R vectors — consumes roughly 100 GiB for a full 10-partition PoRep proof. On a machine with 256 GiB of RAM, this means the pipeline queue can hold at most one pre-synthesized proof at a time.

The design document explored a crucial optimization: processing partitions individually rather than all 10 at once. The existing CompoundProof::circuit_proofs() method batches all partitions together, but the design realized that since each partition's circuit is independent, they could be synthesized one at a time, reducing the peak intermediate state from ~136 GiB to ~13.6 GiB. This per-partition pipeline strategy became a cornerstone of the Phase 2 architecture.

The document also covered the SRS manager design (how to keep the SRS resident in GPU memory across proofs), the memory budget analysis (how to partition RAM between synthesis results, SRS data, and working space), and a 7-step implementation plan. The assistant then updated the memory analysis after running a verification task (<msg id=404-405>) that confirmed the exact constraint counts from the production constants file.

The Versioning Mistake: A Lesson in Cargo's Semver Semantics

The implementation hit a snag that reveals an important assumption about Rust's package management. When the assistant initially set the fork's version to 0.26.0-cuzk.1 ([msg 413]), the [patch.crates-io] section in the workspace Cargo.toml failed to activate it. The warning message was clear: "Patch bellperson v0.26.0-cuzk.1 was not used in the crate graph."

The root cause is Cargo's semver compatibility rules. The dependency specification in the transitive chain (from filecoin-proofs-api through to bellperson) was bellperson = &#34;0.26.0&#34;, which in semver means &gt;=0.26.0, &lt;0.27.0. A pre-release version like 0.26.0-cuzk.1 has a lower precedence than 0.26.0 — it is not considered compatible with the 0.26.0 requirement. The assistant corrected this by changing the version to exactly 0.26.0 ([msg 422]), which satisfied the dependency and allowed Cargo to substitute the local fork for the upstream crate.

This was a subtle mistake with a straightforward fix, but it illustrates the kind of friction that arises when modifying third-party dependencies in a complex workspace. The assistant's debugging process — reading the warning, reasoning about semver rules, applying the fix, and verifying with cargo update -p bellperson — is a textbook example of systematic troubleshooting.

What the Commit Actually Represents

When the assistant runs git add extern/bellperson/ extern/cuzk/Cargo.toml extern/cuzk/Cargo.lock cuzk-phase2-design.md, it is committing the entire foundation for Phase 2. The extern/bellperson/ directory contains the forked library with the exposed APIs. The Cargo.toml and Cargo.lock changes wire the workspace to use the local fork via [patch.crates-io]. The cuzk-phase2-design.md document captures the architectural reasoning and implementation plan.

But the commit also represents something more intangible: the transition from design to implementation. Before this commit, the pipelined prover existed only as a concept in design documents and todo lists. After this commit, the API surface exists in compiled, tested, committed code. The next step — implementing the actual pipelined prover in cuzk-core — can proceed against a real, compilable foundation.

The assistant's brief comment "Good, no target directory" is a small validation check — confirming that the bellperson fork's build artifacts haven't been accidentally included in the commit. This attention to detail (checking for target/ directories before committing) reflects a disciplined approach to version control that prioritizes clean, minimal commits.

Assumptions and Their Validity

Several assumptions underpin this work, and most are well-justified:

The split is purely a visibility problem. The assistant assumes that the existing internal APIs in bellperson are sufficient for the pipelined architecture — that synthesize_circuits_batch() produces a ProvingAssignment that can be serialized, transferred, and consumed by prove_from_assignments() on a different machine. This is correct for the supraseal backend, where assignments use raw Scalar values (32 bytes each) rather than the Repr format used in the native backend. The design document explicitly notes this distinction.

The per-partition pipeline reduces memory. The assumption that partitions can be processed independently is validated by the circuit structure — each partition's constraints are independent of the others, so there is no cross-partition state that must be preserved. This is a property of the Filecoin proof construction, not a general property of Groth16 proofs.

The [patch.crates-io] mechanism works for transitive dependencies. This is a well-known Cargo feature, but its correct operation depends on version compatibility. The initial version mismatch proved that the assumption "just change the version string" was insufficient — the version must satisfy the semver range specified by the dependent crates.

The Thinking Process: From Analysis to Commit

Tracing the assistant's reasoning across the messages leading to this commit reveals a consistent pattern: analyze first, design second, implement third, verify fourth. The analysis in [msg 400] identified the exact API surface that needed exposure. The design document in [msg 403] mapped the memory landscape and validated the per-partition approach. The implementation in <msg id=410-418> made the minimal changes with careful attention to visibility boundaries. The verification in <msg id=423-425> confirmed compilation and test passing.

The commit in [msg 427] is the natural conclusion of this process — the point where all the pieces are assembled and the work is checkpointed. It is not the end of Phase 2 (the pipelined prover implementation in cuzk-core remains to be written), but it is the end of the foundation. Everything from here forward builds on committed, tested code.

Conclusion

Message [msg 427] is a commit message, but it is also a milestone marker. It captures the moment when a carefully reasoned architectural plan materialized as compilable, tested code. The 791-line design document, the 130-line bellperson fork, the workspace patch configuration — all of these represent the output of a rigorous engineering process that prioritized understanding over haste, minimalism over rewriting, and verification over assumption. The commit itself is two lines of shell commands, but the thinking behind it spans dozens of messages and represents the difference between a good idea and a working system.