Mapping the Groth16 Proving Pipeline: A Systematic Investigation of Filecoin's Proof Generation Architecture

Introduction

In the architecture of Filecoin's proof-of-replication (PoRep) system, the path from a high-level sealing API to the low-level Groth16 prover is anything but straightforward. The codebase is distributed across multiple Rust crates—filecoin-proofs-api, filecoin-proofs (v1), storage-proofs-core, storage-proofs-porep, storage-proofs-post, storage-proofs-update, and finally bellperson—each adding its own layer of abstraction, type parameterization, and orchestration logic. Understanding how these layers connect is not merely an academic exercise: it is a prerequisite for any serious optimization effort, particularly when the goal is to reduce the ~200 GiB peak memory footprint of the SUPRASEAL_C2 pipeline or to bypass entire layers for a custom prover.

This article synthesizes a deep-dive investigation that systematically traced every link in this chain. The work, conducted across a multi-round opencode session, produced a comprehensive map of the Groth16 proof generation pipeline for all three Filecoin proof types—PoRep (seal), PoSt (Proof-of-Spacetime), and EmptySectorUpdate—from the top-level API entry points down to the CUDA kernel dispatch in bellperson's prover module. Along the way, it identified every circuit struct, every CompoundProof wrapper, every batch-size constant, and every decision point, culminating in a practical assessment of whether the filecoin-proofs layer can be bypassed when calling bellperson directly from a custom proving pipeline.

The Investigation Begins: A Precision Request

The investigation was launched by a user request of extraordinary specificity ([msg 0]). The user asked to trace exactly how filecoin-proofs-api::seal::seal_commit_phase2 reaches bellperson's create_proof_batch_priority, and to do the same for the PoSt and EmptySectorUpdate proof paths. This was not idle curiosity—the broader context, revealed by the analyzer summary, shows that this investigation was part of a larger project to understand and optimize the SUPRASEAL_C2 Groth16 proof generation pipeline, which consumes approximately 200 GiB of peak memory during proof generation.

The user's request listed four specific files to examine, covering the full call chain from the high-level API through the inner implementation to the core proving framework and finally the circuit definitions. The user also asked about PoSt and Update paths, demonstrating an understanding that any optimization to the proof generation pipeline must account for all three proof types used by the Filecoin protocol. Crucially, the user asked whether the filecoin-proofs layer could be bypassed entirely when calling bellperson directly from a custom prover called "cuzk"—a question that would drive the investigation's most practically valuable analysis.

The Method: Systematic Source Code Archaeology

The assistant's response to this request unfolded as a masterclass in methodical codebase investigation. Rather than guessing or relying on documentation, the assistant systematically located and read source files across the entire dependency chain, using a combination of glob, bash, cat, rg (ripgrep), and find commands.

Phase 1: Locating the Source Files

The first step was to locate the relevant source files in the Cargo registry cache ([msg 1], [msg 2]). The assistant issued glob commands to find files matching the user's specified paths, then used bash with ls to confirm their existence and resolve the registry hash. This revealed that all four packages—filecoin-proofs-api-19.0.0, filecoin-proofs-19.0.1, storage-proofs-core-19.0.1, and storage-proofs-porep-19.0.1—were stored under the same registry hash 1949cf8c6b5b557f, confirming they were fetched in the same Cargo operation.

Phase 2: Reading the Upper Layers

With file paths confirmed, the assistant read the critical source files ([msg 3]). Four files were read in parallel: the outer seal.rs from filecoin-proofs-api, the compound_proof.rs from storage-proofs-core, and two circuit definition files from storage-proofs-porep's stacked circuit module (params.rs and proof.rs). This top-down sampling strategy revealed the key architectural pattern: seal_commit_phase2 in the API layer delegates to an inner implementation via the with_shape! macro, which calls CompoundProof::prove_with_vanilla() from storage-proofs-core. That generic method is implemented by each proof type's CompoundProof wrapper, which constructs circuit instances and invokes bellperson's batch proof creation.

The assistant then read the inner implementation layer ([msg 4]), filecoin-proofs/src/api/seal.rs, which confirmed how the v1 implementation bridges the public API to the proof-system abstractions. This layer handles partition setup, tree construction, vanilla proof generation, and Groth16 parameter loading from disk caches.

Phase 3: Expanding to PoSt and Update Paths

Having established the PoRep call chain, the assistant pivoted to the PoSt and Update paths ([msg 5]). Directory listings revealed the file structure: filecoin-proofs-api/src/ contained post.rs and update.rs, while filecoin-proofs/src/api/ contained winning_post.rs, window_post.rs, and update.rs. The assistant read the outer API files ([msg 6]) and then the inner implementation files ([msg 7]), confirming that all three proof types follow the same architectural pattern.

The assistant then drilled into the circuit definitions for PoSt and Update ([msg 8], [msg 9], [msg 10]). The PoSt circuit lives in storage-proofs-post/src/fallback/circuit.rs as FallbackPoStCircuit<Tree>, with its compound proof wrapper FallbackPoStCompound<Tree> in compound.rs. The Update circuit lives in storage-proofs-update/src/circuit.rs as EmptySectorUpdateCircuit<TreeR>, with EmptySectorUpdateCompound<TreeR> in compound.rs. Both follow the same pattern as PoRep's StackedCircuit and StackedCompound.

The assistant even verified the Window PoSt path ([msg 11]), confirming that it uses the same FallbackPoStCompound and CompoundProof machinery as Winning PoSt. This thoroughness ensured that the mental model being constructed was comprehensive and accurate.

The Bellperson Boundary: Tracing into the Prover

With the upper layers mapped, the assistant turned to bellperson itself—the Groth16 proving library where the actual GPU-accelerated proof computation occurs.

Locating Bellperson

The assistant first located the bellperson source directory ([msg 12]), finding two copies in different registry snapshots. Choosing the one in the same registry directory as the other Filecoin packages ensured version consistency. A targeted search for the public API function ([msg 13]) revealed that create_random_proof_batch_in_priority appears in two files: ext.rs and ext_supraseal.rs. This was a significant finding—it hinted at a bifurcated architecture with both standard and GPU-accelerated proving paths.

Confirming the Function Signatures

The assistant verified the exact function signatures ([msg 14], [msg 15]). The standard variant in ext.rs takes a generic P: ParameterSource<E> parameter, while the supraseal variant in ext_supraseal.rs takes a concrete &SuprasealParameters<E>. Both are thin wrappers that delegate to the internal create_random_proof_batch_priority with priority=true. The generic constraints reveal critical type requirements: circuits must implement Circuit<E::Fr> + Send, parameters must satisfy ParameterSource<E>, and the field and group elements must be GPU-compatible (GpuName).

Drilling into the Implementation

The assistant then traced into the internal implementation ([msg 16], [msg 17], [msg 18], [msg 19]). A broad search across the bellperson source tree revealed that create_random_proof_batch_priority appears in three files: ext.rs (call site), ext_supraseal.rs (supraseal variant), and critically, prover/mod.rs (the actual implementation). The prover/mod.rs function is marked pub(super), meaning it is accessible only within the groth16 module. Its body begins by generating random blinding factors (r_s and s_s values) for each circuit—the first step in the Groth16 prover's randomized exponentiation.

Verifying the Re-export Chain

Before delivering the final analysis, the assistant performed a critical verification step ([msg 20], [msg 21], [msg 22]). The storage-proofs-core code imports create_random_proof_batch_in_priority as bellperson::groth16::create_random_proof_batch_in_priority. The assistant needed to confirm that the function is actually re-exported at that path. Searching groth16/mod.rs for pub use statements revealed that both ext::* and ext_supraseal::* are re-exported via wildcard imports, confirming the public API surface.

The assistant then examined the supraseal variant in detail ([msg 23]), discovering that it takes &SuprasealParameters<E> instead of a generic ParameterSource. This streamlined interface is exactly what a custom proving pipeline could use to call bellperson directly, without going through the filecoin-proofs or storage-proofs-core layers.

The Synthesis: A Complete Call Chain Map

The culmination of the investigation was a comprehensive synthesis ([msg 24]) that organized the scattered source code findings into a structured analysis. The response contained five major sections.

Call Chain Diagrams

The most visually striking element was the set of ASCII call chain diagrams tracing seven levels of function calls for each proof type. For the Seal (PoRep) path, the chain is:

  1. filecoin_proofs_api::seal::seal_commit_phase2() — the top-level API entry point
  2. with_shape!() — the macro that dispatches on sector size to select the concrete Merkle tree type
  3. filecoin_proofs_api::seal::seal_commit_phase2_inner::<Tree>() — the type-erased inner function
  4. filecoin_proofs_v1::seal_commit_phase2::<Tree>() — the v1 implementation that builds public inputs and loads Groth16 parameters
  5. StackedCompound::<Tree, DefaultPieceHasher>::circuit_proofs() — the default CompoundProof trait implementation that iterates partitions and builds circuit instances
  6. bellperson::groth16::create_random_proof_batch[_in_priority]() — the bellperson entry point
  7. bellperson::groth16::prover::create_random_proof_batch_priority() and create_proof_batch_priority_inner() — the actual prover internals The diagrams for Winning PoSt, Window PoSt, and Sector Update follow the same pattern, revealing that they all converge at the same bellperson functions through the same CompoundProof::circuit_proofs() default trait method.

Circuit Struct Identification

The assistant precisely identified three circuit structs:

Circuit Construction Process

The assistant explained how circuits are constructed from vanilla proof outputs. The CompoundProof::circuit_proofs() method iterates over partitions, calling each proof type's circuit() method. For PoRep, StackedCompound::circuit() extracts comm_r_last and comm_c from the vanilla proof, then maps each vanilla proof to a circuit Proof via the From<VanillaProof> implementation. The resulting StackedCircuit holds {replica_id, comm_d, comm_r, comm_r_last, comm_c, proofs: Vec<Proof>}. When bellperson calls synthesize(), the circuit allocates public inputs, verifies the commitment relationship, and for each challenge proof synthesizes the DRG/expander parent column proofs, labeling, and inclusion proofs.

Layer Stack Analysis

The assistant identified four distinct layers between the top-level API and bellperson:

  1. filecoin-proofs-api — type-erased API layer, dispatches via with_shape! macro
  2. filecoin-proofs (v1) — concrete Tree-parameterized API, loads Groth16 parameters, builds public/private inputs
  3. storage-proofs-core — CompoundProof trait with default circuit_proofs() implementation, batch management, priority dispatch
  4. storage-proofs-{porep,post,update} — Circuit<Fr> implementations, circuit constructors The critical insight is that CompoundProof::circuit_proofs() in storage-proofs-core is the single junction point where vanilla proofs are converted into circuit instances and dispatched to bellperson. All four proof types share the same default trait method implementations, differing only in their circuit(), blank_circuit(), and generate_public_inputs() implementations.

Bypass Feasibility Analysis

The final and most practically valuable section addressed whether the filecoin-proofs layer could be bypassed. The assistant's analysis was nuanced: yes, in principle, you can bypass filecoin-proofs-api and filecoin-proofs, but you cannot avoid the circuit construction step in storage-proofs-* without reimplementing the entire circuit witness generation. The practical recommendation is a thin Rust shim that deserializes vanilla proofs, loads Groth16 parameters, calls CompoundProof::circuit() to build circuit instances, and then feeds them to a custom prover.

Key Findings and Their Significance

Several findings from this investigation have lasting significance for the SUPRASEAL_C2 optimization project.

The Batch Size and Priority Mechanism

The investigation documented two important operational parameters: MAX_GROTH16_BATCH_SIZE = 10 and the priority flag that selects between create_random_proof_batch and create_random_proof_batch_in_priority. A batch size of 10 means that even with many partitions, circuits are processed in groups of 10, which affects GPU utilization and memory usage. The priority flag provides a mechanism for influencing prover scheduling behavior.

The Two Bellperson Paths

The discovery of both ext.rs and ext_supraseal.rs modules in bellperson revealed a bifurcated architecture. The standard path uses a generic ParameterSource<E> trait, while the supraseal path uses a concrete &SuprasealParameters<E>. This means that a custom proving pipeline can choose the simpler supraseal interface if it can construct the SuprasealParameters struct, which bundles the proving key, verifying key, and SRS into a single pre-loaded structure.

The Shared CompoundProof Infrastructure

All four proof types (PoRep, Winning PoSt, Window PoSt, Sector Update) use the same CompoundProof::circuit_proofs() machinery. This means that optimizations applied at this level benefit all proof types simultaneously. A memory optimization in the batch processing logic, for example, would reduce memory for all four proof types.

The Critical Circuit Construction Dependency

The circuit construction step cannot be easily removed or replaced. The circuit() method in each CompoundProof implementation converts vanilla proof data structures into the circuit's private witness values, and this conversion is deeply tied to the storage-proofs-* crate types. Reimplementing it would mean reimplementing the entire circuit witness generation—a substantial undertaking given that the PoRep circuit alone involves DRG parent column proofs, labeling verification, encoding verification, and multiple Merkle inclusion proofs.

Conclusion

This investigation transformed scattered source code across six Rust crates into a coherent, structured map of the Filecoin Groth16 proving pipeline. The work identified every circuit struct, every trait implementation, every batch-size constant, and every decision point along the way. It provided a concrete bypass feasibility analysis that directly informs the architecture of a custom prover. And it did all of this with evidence-based reasoning, cross-referencing, and strategic thinking.

For anyone working on Filecoin proof optimization—whether reducing memory, improving throughput, or replacing the prover entirely—this analysis is an indispensable reference. It answers not just the question that was asked, but the questions that follow: where to intervene, what to keep, what to discard, and how the pieces connect. In a codebase as layered and complex as Filecoin's proof pipeline, that kind of clarity is worth its weight in gold.