The Anatomy of a Deep-Dive Investigation: Mapping Filecoin's SUPRASEAL_C2 Groth16 Proof Pipeline

Introduction

In the world of decentralized storage networks, few systems push the boundaries of computational intensity quite like Filecoin's Proof-of-Replication (PoRep) pipeline. At its peak, a single proof generation consumes approximately 200 GiB of memory, spans code written in Go, Rust, C++, and CUDA, and involves cryptographic operations ranging from SHA-256 hashing to Groth16 zk-SNARK proving over the BLS12-381 elliptic curve. Understanding this pipeline—let alone optimizing it—requires a level of systematic investigation that borders on forensic analysis.

This article examines a single chunk from an opencode coding session that undertook precisely such an investigation. The chunk represents a deep-dive into the SUPRASEAL_C2 Groth16 proof generation pipeline for Filecoin PoRep, driven by the goal of understanding and dramatically reducing its ~200 GiB peak memory footprint. Over the course of dozens of messages spanning multiple subagent sessions, the investigation produced four comprehensive documents: a background reference mapping the entire call chain with file:line references and nine identified structural bottlenecks, plus three composable optimization proposals. The overarching achievement was a shift in perspective—from optimizing individual proof generation toward architecting a continuous, memory-efficient proving pipeline optimized for heterogeneous cloud rental markets where RAM cost dominates.

This article synthesizes the work across the entire chunk, tracing the investigation's trajectory from initial research probes through deep code analysis to the final optimization proposals and micro-architectural insights.

The Opening Salvo: Systematic Research as a Foundation

The investigation began with a single, meticulously crafted user message (see [msg 0]) that laid out eight specific research targets spanning multiple Rust crates in the Cargo registry. The user was implementing "Phase 2 of cuzk"—a pipelined proving engine designed to replace the monolithic PoRep C2 prover with a per-partition synthesis/GPU architecture. The eight questions ranged from finding the PoRepConfig struct definition to understanding how MultiProof::write() assembles final proof bytes, from discovering type aliases like SectorShape32GiB to tracing the parameter cache file-naming convention.

What makes this opening message remarkable is not just its specificity but its strategic discipline. The user explicitly closed with "Research only," constraining the assistant to information gathering and preventing premature code generation. This was a deliberate workflow control—a recognition that in a system handling ~200 GiB of peak memory across multiple language boundaries, skipping the research phase would be catastrophic.

The assistant's response was equally methodical. In the first round ([msg 1]), it launched six parallel glob searches to locate the relevant source files across the Cargo registry. This was a foundational step: without knowing where the code lives, no detailed answers can follow. The glob patterns were carefully chosen—some highly specific (targeting parameters.rs directly), some broad (a catch-all for all source files in the filecoin-proofs crate), and some intermediate (module entry points, type definition files). This layered approach balanced precision with completeness, ensuring that no critical file would be missed.

The Research Phase: Parallel Source Code Analysis

With the file locations established, the assistant moved to the data-gathering phase. In [msg 2], it issued eight parallel read tool calls, each targeting a specific source file that contained answers to one or more of the user's questions. The files read spanned the entire proof pipeline:

Drilling Deeper: grep as a Surgical Instrument

After the initial file reads, the assistant transitioned to targeted symbol hunting using grep ([msg 3]). This represented a shift from "where are the files?" to "what exactly do these functions do?" The assistant issued five grep searches in parallel, each targeting a different aspect of the codebase:

  1. as_safe_commitment — A helper function that converts randomness bytes into a domain element for the proof circuit
  2. seal_commit_phase2 — The core function orchestrating Phase 2 proof generation
  3. cache_prefix — The function generating unique prefix strings for parameter cache files
  4. struct Challenges|new_interactive|new_non_interactive|new_synthetic|num_challenges_per_partition — A complex regex pattern to find the challenge configuration The grep results were rich. The as_safe_commitment search returned 37 matches across the codebase. The seal_commit_phase2 search located the function in api/seal.rs at lines 503-590. The cache_prefix search found 44 matches, including the critical implementation for StackedCompound in storage-proofs-porep-19.0.1/src/stacked/circuit/proof.rs:185-189, which revealed the format string: "stacked-proof-of-replication-{}-{}" with Tree::display() and G::name(). The Challenges search returned 117 matches, revealing the Challenges struct and its construction methods. One of the most valuable discoveries was the cache_prefix implementation, which showed that the parameter file naming convention was deterministic and test-asserted. The test at line 1295 of params.rs statically asserted identifiers for specific tree types, with a comment explicitly stating: "The identifier is used for the parameter file filenames. It must not change, as the filenames are fixed for the official parameter files. Hence staticly assert certain identifiers." This was a strong guarantee that simplified the pipeline's parameter loading logic.

Reading the Remaining Critical Files

With the grep results in hand, the assistant read the remaining critical files in [msg 4]. Six files were read in parallel, each selected to fill specific gaps in the investigation:

Tracing the Parameter Identity Chain

A significant portion of the investigation focused on understanding how parameter cache filenames are constructed—a critical piece of infrastructure for the Phase 2 pipeline. The chain of identity construction was traced through multiple messages:

In [msg 5], the assistant read the SetupParams struct and searched for Tree::display() and identifier() methods. The key discovery was the test at lines 1295-1300 of params.rs, which statically asserted that the identifier for OctTree32Gib (a DiskTree<PoseidonHasher, ...>) matched a specific string. This test encoded the protocol's invariant: the identifier must never change, as it determines the filenames of official parameter files.

In [msg 6], the assistant read the actual implementations of MerkleTreeTrait::display() and StackedGraph::identifier(). The display() method was documented as "Print a unique name for this configuration," and the identifier() method simply returned self.id.clone(), where self.id was a pre-computed string constructed during StackedGraph::new(). This told the assistant that the identifier was computed once and stored, not generated on-the-fly.

In [msg 7], the assistant searched for Hasher::name() implementations, confirming that PoseidonHasher::name() returned "poseidon_hasher" and Sha256Hasher::name() returned "sha256_hasher". These concrete string values were essential for reconstructing cache filenames.

In [msg 8], the assistant read the StackedGraph::new() constructor to see how the id field was built, and confirmed the hasher name implementations. The id construction involved a format!() call combining the base graph identifier, node count, hasher name, and other parameters.

The complete chain was now mapped: cache_prefix() produced "stacked-proof-of-replication-{Tree::display()}-{G::name()}", which for 32 GiB sectors became "stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher". This prefix was combined with a SHA-256 hash of pub_params.identifier() to produce the final filename: /var/tmp/filecoin-proof-parameters/v28-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-<SHA256_HEX>.params.

A critical observation emerged: the identifier string depended only on the graph structure, not on the porep_id. This meant that all 32 GiB proof types (V1, V1_1, Synthetic) that used 18 challenges per partition shared the same parameter file. This had important performance implications—the parameter file (approximately 1.1 GiB for 32 GiB sectors) only needed to be loaded once, even if the proof type changed.

The Synthesis: A Comprehensive Map of the Proof Pipeline

The culmination of the research phase was [msg 9], where the assistant compiled all findings into a comprehensive answer covering all eight user questions. This message was remarkable for its density of technical information and for the way it wove together details from half a dozen separate source files into a coherent narrative.

The eight findings formed a logical progression from the highest-level configuration down to the lowest-level file path resolution:

  1. PoRepConfig struct definition — Five fields: sector_size, partitions, porep_id, api_version, api_features. The partitions count was looked up from a global POREP_PARTITIONS map, allowing runtime overrides.
  2. as_v1_config() for 32 GiB sectors — For StackedDrg32GiBV1_1: sector_size = 34359736832, partitions = 10, porep_id = [0x08, 0, 0, ...], api_version = V1_1_0. For non-interactive PoRep: partitions = 126—a 12.6× increase with profound memory implications.
  3. SectorShape32GiB type alias — Resolved to LCTree<PoseidonHasher, U8, U8, U0>, a Poseidon-hashed Merkle tree with base arity 8, sub-tree arity 8, and no top tree. The display string was "merkletree-poseidon_hasher-8-8-0".
  4. MultiProof::new() and MultiProof::write() — Each Groth16 proof was exactly 192 bytes (two G1 points and one G2 point in BLS12-381 compressed form). For 10 partitions, the total proof was 1920 bytes.
  5. as_safe_commitment and DefaultPieceDomain — The function converted 32-byte commitments into field elements, validating they were less than the BLS12-381 field modulus. An asymmetry was noted: comm_r used as_safe_commitment (generic over Domain), while comm_d used DefaultPieceDomain::try_from_bytes directly.
  6. setup_params function — For 32 GiB interactive PoRep: nodes = 1073741824 (over 1 billion), degree = 6, expansion_degree = 8, challenges = Challenges::Interactive(18), num_layers = 11.
  7. LAYERS and select_challenges — LAYERS returned 11 for 32 GiB. select_challenges computed 18 challenges per partition (176 minimum / 10 partitions, rounded up). The "8-8-0" terminology was clarified as referring to tree arity, not challenge counts.
  8. Param file path convention — The complete chain from cache_prefix() through cache_identifier() to the final file path was traced, revealing the shared parameter file across proof types.

From Understanding to Optimization: The Three Proposals

The investigation did not stop at understanding—it produced actionable optimization proposals. Building on the detailed memory accounting that showed exactly where each GiB went (10 parallel partition circuits at ~16 GiB each, plus ~48 GiB SRS in pinned memory), three composable proposals were developed:

Proposal 1: Sequential Partition Synthesis — This broke the all-10-partitions-in-parallel model by streaming partitions one-at-a-time through the GPU. This reduced peak memory from ~200 GiB to approximately 64-103 GiB, enabling the pipeline to function on machines with 128 GiB of RAM. The key insight was that partitions were independent—each produced its own Groth16 proof, and the MultiProof structure naturally accepted a Vec<groth16::Proof>. There was no cryptographic reason to synthesize all partitions simultaneously.

Proposal 2: Persistent Prover Daemon — This eliminated the ~60-second per-proof SRS loading overhead by keeping the proving process alive across proofs. The shared parameter file across all 32 GiB proof types meant that the SRS only needed to be loaded once. A persistent daemon could accept new proof jobs without reloading parameters, dramatically reducing per-proof latency.

Proposal 3: Cross-Sector Batching — This exploited the freed memory headroom from Proposal 1 to batch multiple sectors' circuits into single GPU invocations. By processing multiple sectors' proofs in a single GPU run, the proposal projected 2-3× throughput per GPU and 5-6× reduction in cost per proof when combined with the other proposals.

Micro-Optimization Analysis: The Final Layer

The final round of analysis examined computational hotpaths at the instruction level. The CPU synthesis hotpaths were characterized: the enforce() loop in circuit synthesis, SHA-256 bit manipulation for Merkle tree hashing, and Fr field arithmetic for constraint evaluation. The GPU NTT (Number Theoretic Transform) and MSM (Multi-Scalar Multiplication) compute characteristics were analyzed, revealing memory-bandwidth-bound patterns. The H-to-D (host-to-device) transfer patterns were mapped, showing where data movement between CPU and GPU memory dominated latency.

One of the most intriguing analyses examined the feasibility of recomputing the a/b/c vectors on-the-fly rather than materializing them in memory. The a/b/c vectors are the linear combination coefficients that represent the circuit's satisfiability—they are typically computed during synthesis and stored for the prover. The analysis considered whether these vectors could be recomputed from the circuit structure during GPU proving, avoiding the memory cost of storing them. This was a radical idea that would trade compute (re-synthesizing) for memory (not storing), and the analysis provided the data needed to evaluate the tradeoff.

The Overarching Achievement: A Shift in Perspective

What makes this chunk truly remarkable is not just the depth of the analysis but the shift in perspective it represents. The investigation began with a narrow question—how does the SUPRASEAL_C2 pipeline work?—and ended with a fundamentally different approach to proof generation. The three optimization proposals, when taken together, represent a move from optimizing individual proof generation toward architecting a continuous, memory-efficient proving pipeline.

This shift was driven by a clear-eyed assessment of the economics of cloud rental markets. In environments where RAM cost dominates the total cost of operation, the ability to reduce peak memory from ~200 GiB to ~64-103 GiB (Proposal 1) is not just a technical improvement—it is a business advantage. The ability to eliminate SRS loading overhead (Proposal 2) and batch multiple sectors (Proposal 3) compounds this advantage, potentially reducing cost per proof by 5-6×.

The investigation also produced a detailed background reference document that mapped the entire call chain from Curio's Go task layer through Rust FFI into C++/CUDA kernels, with file:line references for every significant function. This document, combined with the memory accounting that showed exactly where each GiB went, provides a foundation that future optimization work can build upon.

Conclusion

The deep-dive investigation into the SUPRASEAL_C2 Groth16 proof generation pipeline for Filecoin PoRep represents a model of systematic codebase analysis. From the initial research manifesto that laid out eight precise questions, through the parallel glob searches and targeted grep operations, to the final synthesis of findings and optimization proposals, every step was methodical and deliberate.

The investigation produced concrete, actionable knowledge: the complete parameter cache filename convention, the exact tree shape for 32 GiB sectors, the proof serialization format, the memory accounting for each component of the pipeline, and three composable optimization proposals with projected performance improvements. But more importantly, it produced a shift in perspective—from optimizing a single proof generation to architecting a continuous, memory-efficient proving pipeline.

For anyone working on Filecoin proof generation, whether optimizing the existing pipeline or building new tools, this investigation provides an invaluable reference. It answers the questions that are essential for understanding, debugging, and optimizing the system: What are the concrete values for a 32 GiB sector? How are proofs serialized? Where are parameter files stored? Where does the memory go? And most importantly, how can we make it better?