The Pivot: How One Message Transformed a Proving Engine's Architecture

Introduction

In the course of building a pipelined SNARK proving engine called cuzk for Filecoin's Groth16 proof generation, a single message at index 395 marks the critical turning point between Phase 1 and Phase 2 of the project. This message is deceptively brief — just a statement of intent and a subagent task dispatch — but it represents the moment where the assistant transitions from implementation mode to deep research mode, and in doing so, discovers the key insight that makes the entire Phase 2 architecture possible.

The message reads, in its entirety:

Phase 2 requires understanding exactly how bellperson works internally to know what API to expose. Let me study the bellperson prover code in detail. [task] {"description":"Analyze bellperson prover internals",...}

What follows is a subagent task result containing a complete analysis of bellperson's Groth16 prover internals. This single message, and the analysis it triggers, unlocks the realization that the synthesis/GPU split the project needs already exists internally — it just needs to be exposed.

The Broader Context: Phase 1 Complete, Phase 2 Beckons

To understand why this message matters, we need to understand where the project stood at this moment. The cuzk proving engine had just completed Phase 1, committed as 9d8453c3. Phase 1 delivered:

A gRPC-based proving daemon that could accept proof requests over the network
Support for all four Filecoin proof types: WinningPoSt, WindowPoSt, SnapDeals, and PoRep C2
A multi-GPU worker pool with priority scheduling and CUDA_VISIBLE_DEVICES isolation
A gen-vanilla command for generating vanilla proof test data from sealed sector data on disk
Observability infrastructure including tracing spans, Prometheus metrics, and graceful shutdown Phase 1 was essentially a wrapper around the existing filecoin-proofs-api — it took the monolithic proof generation pipeline and made it accessible over gRPC, but it didn't change the internal architecture of how proofs were generated. The proof pipeline remained a single, indivisible operation: submit a proof request, wait for the GPU, get back a proof. Phase 2, as defined in the project plan (cuzk-project.md), had a much more ambitious goal: split the proof generation pipeline into two independently schedulable phases — CPU-bound circuit synthesis and GPU-bound computation (NTT/MSM). This split would enable: 1. Pipelining: Synthesis for sector N+1 could begin while GPU computes proof for sector N 2. Memory reduction: Intermediate state (the synthesized circuit assignments) could be streamed partition-by-partition rather than held in memory all at once 3. SRS residency: The Structured Reference String (SRS) could be loaded once and kept in GPU memory across multiple proofs, eliminating the dominant overhead in the current pipeline But before any of this could be implemented, the assistant needed to understand one thing: does bellperson already have a clean internal boundary between synthesis and GPU computation? The answer would determine whether Phase 2 required a massive refactor of bellperson itself, or whether a minimal fork would suffice.

Why This Message Was Written: The Reasoning and Motivation

The assistant's explicit motivation is stated in the first sentence: "Phase 2 requires understanding exactly how bellperson works internally to know what API to expose." This is a recognition that the project is about to touch a critical dependency — bellperson is the Rust crate that implements Groth16 proving for Filecoin, and it sits at the very heart of the proof generation pipeline. Modifying it without understanding its internals would be reckless.

But there's a deeper reasoning at play here. The assistant has been following a deliberate pattern throughout the cuzk project: investigate thoroughly before modifying. Earlier in the conversation, when designing the Phase 0 gRPC API, the assistant studied the existing filecoin-proofs-api signatures, the protobuf definitions, and the Curio integration layer before writing any code. When implementing gen-vanilla, the assistant traced the call chain from seal_commit_phase2 through CompoundProof::circuit_proofs to understand exactly how vanilla proofs were generated. This pattern of research-first, implementation-second is a hallmark of the assistant's approach.

The motivation is also strategic. The assistant knows that forking bellperson is a significant decision — it creates a maintenance burden, requires patching the workspace's dependency graph, and potentially diverges from upstream. A fork should only be undertaken if the changes are minimal and well-understood. By analyzing bellperson's internals first, the assistant can determine the minimum viable fork — the smallest set of changes that enables the Phase 2 architecture.

The Task Dispatch Pattern: How Subagents Enable Deep Analysis

The message uses the task tool to spawn a subagent. This is a distinctive feature of the opencode session architecture: rather than reading files one at a time in the main conversation, the assistant can delegate an entire research thread to a subagent that runs to completion and returns a synthesized result.

The subagent's prompt requests "very thorough" analysis and specifies the exact goal: "understand bellperson's Groth16 prover internals to design a split synthesis/GPU API." The subagent is instructed to read specific files, trace call chains, and identify the exact boundary between the CPU synthesis phase and the GPU computation phase.

This pattern is powerful because it allows the assistant to parallelize deep investigation. While the subagent is reading dozens of source files and tracing function calls, the main conversation thread is freed to continue other work. In this case, the subagent's analysis returns inline in the same message, so the assistant can act on it immediately in the next round.

Input Knowledge Required

To fully understand this message, one needs:

Knowledge of the cuzk project architecture: That Phase 1 has just completed, that Phase 2 aims to split synthesis from GPU computation, and that the project plan calls for a bellperson fork.
Understanding of Groth16 proof generation: That the pipeline consists of two distinct phases — circuit synthesis (running each circuit's synthesize() method to produce constraint assignments) and the GPU phase (performing Number Theoretic Transforms and Multi-Scalar Multiplications to produce the actual proof).
Knowledge of the dependency graph: That filecoin-proofs-api calls into filecoin-proofs, which calls into storage-proofs-core (for CompoundProof::circuit_proofs), which calls into bellperson (for create_random_proof_batch). Understanding this chain is essential for knowing where the split point should be.
Awareness of the SRS loading problem: That the current pipeline loads the Structured Reference String (SRS) from disk into GPU memory for every proof, consuming ~30 seconds of overhead per proof. The Phase 2 design aims to keep the SRS resident in GPU memory across proofs.
Understanding of the task tool: That it spawns a subagent that runs to completion and returns its results inline. The assistant cannot act on the subagent's results until the next round.

Output Knowledge Created

The subagent's analysis, which is the core output of this message, produces several critical pieces of knowledge:

The discovery that synthesize_circuits_batch() already exists as a standalone internal function in bellperson's src/groth16/prover/supraseal.rs. This function performs CPU-only circuit synthesis and returns Vec<ProvingAssignment<Scalar>> — the intermediate state that can be passed to the GPU phase.
The exact boundary between synthesis and GPU computation: The synthesis phase produces ProvingAssignment structs containing a, b, c evaluation vectors and density trackers. The GPU phase takes these assignments, packs them into raw pointers, and calls supraseal_c2::generate_groth16_proof().
The minimal fork requirements: Only three changes are needed — make ProvingAssignment<Scalar> public, make synthesize_circuits_batch() public, and add a new prove_from_assignments() function that runs only the GPU phase.
The realization that no logic changes are needed: The split is already clean. The fork only needs to change visibility modifiers (~30 lines of changes, ~130 lines total including new function).
The understanding that DensityTracker is already public via the ec-gpu-gen dependency, so no additional changes are needed there. This knowledge transforms the Phase 2 implementation from a speculative design into a concrete, achievable plan. The assistant now knows exactly what to fork and what API to expose.

Assumptions Made

The message makes several assumptions, most of which prove correct:

That bellperson has a clean internal boundary between synthesis and GPU computation: This is the central assumption, and it turns out to be correct. The synthesize_circuits_batch() function already exists as a standalone internal function, and the GPU phase code is cleanly separated in create_proof_batch_priority_inner.
That a minimal fork approach is feasible: The assistant assumes that exposing existing internal APIs (rather than rewriting bellperson) is sufficient for Phase 2. This proves correct — the fork requires only ~130 lines of changes, all visibility-related.
That understanding the internal architecture is a prerequisite to implementation: The assistant assumes that it cannot design the Phase 2 pipeline without first understanding bellperson's internals. This is a sound engineering assumption — designing an API without understanding the underlying implementation would risk creating an interface that doesn't match the actual code structure.
That the subagent can perform this analysis effectively: The assistant trusts that the subagent, given a thorough prompt, will read the right files and return a useful analysis. This assumption is validated by the quality of the returned analysis.

The Thinking Process Visible in the Message

While the message itself is brief, the thinking process is visible in several ways:

The explicit statement of purpose: "Phase 2 requires understanding exactly how bellperson works internally to know what API to expose." This shows the assistant's recognition that implementation decisions depend on deep understanding of the existing code.

The choice of the task tool: Rather than reading files manually in the main conversation (which would produce many tool calls and messages), the assistant delegates the entire analysis to a subagent. This shows an understanding of the opencode session's capabilities and a desire to keep the conversation focused.

The thoroughness specification: The subagent prompt requests "very thorough" analysis and specifies the exact files and functions to examine. This shows the assistant's understanding that the quality of the analysis depends on the quality of the prompt.

The task_id for resumability: The subagent is given a task_id that allows it to be resumed if needed. This shows forward-thinking about the possibility of needing to continue the analysis.

What Came After: The Fork and Implementation

The subsequent messages show the assistant acting on the subagent's analysis. In [msg 396], the assistant confirms the key finding: "synthesis and GPU compute are already cleanly separated inside bellperson." It then spawns another subagent to read the exact source files and understand the minimal changes needed.

In [msg 397], the assistant has the complete picture and begins designing the Phase 2 plan. In [msg 398], it writes a detailed analysis of the two possible strategies (fork bellperson only vs. fork multiple crates) and chooses the minimal approach.

By [msg 400], the assistant has traced the full call chain from filecoin-proofs-api through to bellperson and confirmed that CompoundProof::circuit() is public, enabling the split without forking anything except bellperson.

The final result, as documented in the chunk summary, is a 791-line Phase 2 design document (cuzk-phase2-design.md), a bellperson fork with ~130 lines of changes, and a workspace patched via [patch.crates-io] — all building on the foundation laid by this single message.

Conclusion

Message 395 is a pivot point in the cuzk project. It marks the transition from Phase 1 (wrapping the existing monolithic pipeline) to Phase 2 (architecting a split, pipelined proving engine). The message itself is brief — a statement of intent and a subagent dispatch — but it triggers a deep analysis that reveals the key architectural insight: the split the project needs already exists inside bellperson, waiting to be exposed.

This message exemplifies a pattern that recurs throughout the cuzk project: research before implementation, understand before modifying. By investing in deep analysis before writing code, the assistant avoids the risk of designing an API that doesn't match the underlying implementation, and ensures that the bellperson fork is as minimal as possible — just visibility changes, no logic changes.

The message also demonstrates the effective use of the opencode session's subagent capability. Rather than reading dozens of files one at a time in the main conversation, the assistant delegates the entire analysis to a focused subagent, receiving back a synthesized result that can be acted on immediately. This pattern — dispatch analysis, receive synthesis, implement — becomes a template for how the project approaches complex engineering decisions.