The 57-Gigabyte Clue: Filesystem Detective Work in the cuzk Phase 2 Pipeline

In the middle of implementing Phase 2 of the cuzk pipelined proving engine—a substantial refactoring of Filecoin's proof generation infrastructure—the assistant pauses to run a simple ls command. The message is brief, barely a line of reasoning and a single shell invocation, yet it captures a moment of critical investigative reasoning that underpins the entire architecture being built. Message [msg 444] reads in full:

Now I know the exact param filenames. There are also PoSt params - I need to figure out which ones are for winning vs window. Let me get the file sizes to distinguish:

>

`` $ ls -lh /data/zk/params/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-*.params -rw-r--r-- 1 theuser theuser 57G Feb 17 16:05 /data/zk/params/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-0377ded656c6f524f1618760bffe4e0a1c51d5a70c4509eedae8a27555733edc.params -rw-r--r-- 1 theuser theuser 184M Feb 17 16:05 /data/zk/params/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-559e581f022bb4e4ec6e719e563bf0e026ad6de42e56c18714a2c692b1b88d7e.params ``

On its surface, this is nothing more than a developer checking which parameter files exist on disk. But in the context of the cuzk project—a months-long effort to build a high-throughput, memory-efficient SNARK proving daemon for Filecoin—this ls command represents a pivotal moment of knowledge synthesis. The assistant is bridging the gap between abstract code and concrete filesystem reality, and the 57-gigabyte difference between two parameter files becomes the key that unlocks the entire SRS manager design.

The Architecture Context: Why Parameters Matter

To understand why this simple file listing is significant, one must understand what the cuzk Phase 2 pipeline is trying to accomplish. The cuzk proving daemon is a ground-up rearchitecture of how Filecoin proof generation works. In Phase 0 and Phase 1, the daemon was essentially a thin wrapper around the existing filecoin-proofs-api library—it accepted proof jobs over gRPC, dispatched them to GPU workers, and returned results. Each proof call was monolithic: the entire Groth16 proving pipeline (circuit synthesis, NTT, MSM) ran as a single, indivisible unit, with the GPU idling while the CPU synthesized circuits.

Phase 2 aims to break this monolithic pipeline in half. The key insight, documented in the cuzk-phase2-design.md document, is that circuit synthesis (CPU-bound) and the NTT/MSM proving computation (GPU-bound) can be overlapped. While the GPU is proving one partition, the CPU can be synthesizing circuits for the next. This pipelining promises a 1.5–1.8× throughput improvement over Phase 1, but it requires a fundamentally different architecture.

The centerpiece of this new architecture is the SRS manager—a module responsible for loading and managing the Structured Reference String (SRS) parameters that the GPU proving kernels depend on. In the monolithic Phase 0/1 design, SRS parameters were loaded implicitly through a private global cache called GROTH_PARAM_MEMORY_CACHE, hidden inside the filecoin-proofs-api library. The pipeline architecture cannot use this cache because it needs explicit, fine-grained control over which parameters are loaded, when they are loaded, and when they can be evicted to free memory. The SRS manager must know, for each proof type, exactly which .params file on disk corresponds to which CircuitId, so it can call SuprasealParameters::load_from_file() directly.

The Mapping Problem: From CircuitId to Filename

The assistant has already done extensive research into the upstream codebase. In messages [msg 435] through [msg 437], it traced the full call chain from filecoin-proofs-api's seal_commit_phase2 function down through filecoin-proofs and storage-proofs-porep to the actual circuit construction. It discovered the CircuitId enum, which distinguishes between different proof types: PoRep, WinningPoSt, WindowPoSt, and SnapDeals. Each of these circuit types requires a different set of SRS parameters, and each has a corresponding .params file on disk.

But the mapping from CircuitId to filename is not documented anywhere in the code. The parameter files follow a naming convention that encodes the proof type, the merkle tree configuration, and a content hash, but the convention is complex and the assistant cannot simply derive the mapping algorithmically. For example, the PoRep parameter file is named:

v28-stacked-proof-of-replication-merkletree-poseidon_hasher-8-0-0-sha256_hasher-032d3138d22506ec0082ed72b2dcba18df18477904e35bafee82b3793b06832f.params

And the PoSpacetime (Proof-of-Spacetime) files—used for both WinningPoSt and WindowPoSt—follow a different pattern:

v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-<hash>.params

The critical problem is that both WinningPoSt and WindowPoSt use the same "proof-of-spacetime-fallback" circuit family. They share the same naming prefix. The assistant cannot tell from the filename alone which file corresponds to which proof type. This is the puzzle that message [msg 444] sets out to solve.

The Heuristic: File Size as Discriminant

The assistant's reasoning, visible in the message's preamble, is elegant in its simplicity: "I need to figure out which ones are for winning vs window. Let me get the file sizes to distinguish."

This reasoning relies on a deep understanding of how Filecoin's proof system works. WinningPoSt is a simple proof that a miner controls their committed storage at a specific epoch. It involves a small number of challenged sectors and a relatively simple circuit. WindowPoSt, by contrast, proves that a miner is continuously storing all their data over a 24-hour window. It involves many more challenges and a much larger circuit. The size of the SRS parameters—the Structured Reference String that defines the proving system's public parameters—scales with the complexity of the circuit. A larger circuit requires more constraints, more variables, and therefore a larger SRS.

The assistant's hypothesis is that the 57 GiB file corresponds to WindowPoSt (the more complex proof with more partitions) and the 184 MiB file corresponds to WinningPoSt (the simpler proof). This is a reasonable heuristic, but it is not guaranteed. The file size difference is stark—two orders of magnitude—which strongly supports the hypothesis. A 57 GiB parameter file is enormous, consistent with the WindowPoSt circuit's 10-partition structure. A 184 MiB file is modest, consistent with WinningPoSt's single-partition circuit.

Assumptions and Potential Pitfalls

The assistant makes several assumptions in this reasoning that are worth examining critically.

First, it assumes that the two files it found are the only PoSt parameter files, and that one corresponds to WinningPoSt and the other to WindowPoSt. In reality, there could be additional parameter files for other configurations, or the files could correspond to different versions of the same proof type. The assistant's earlier research (message [msg 442]) showed multiple .vk (verification key) files for each proof type, suggesting multiple versions exist. The .params files, however, appear to be unique per circuit family.

Second, the assistant assumes that file size is a reliable proxy for circuit complexity. This is generally true for SRS parameters in Groth16—the parameters contain the proving key, which includes the encoded circuit structure. A larger circuit produces a larger proving key. However, the relationship is not perfectly linear, and there could be other factors affecting file size, such as the number of public inputs or the specific curve parameters.

Third, the assistant assumes that the naming convention is consistent and that the "proof-of-spacetime-fallback" prefix covers both WinningPoSt and WindowPoSt. This is consistent with the codebase structure—both proof types ultimately use the same ProofOfSpacetime circuit family from storage-proofs-post—but the assistant has not yet verified this by examining the actual circuit construction code for each proof type.

These assumptions are reasonable for an investigative step, but they represent knowledge that will need to be validated later, either by tracing the exact parameter loading path in the code or by running actual proofs and observing which file gets loaded.

The Input Knowledge Required

To fully understand this message, a reader needs substantial background knowledge about Filecoin's proof architecture. This includes:

The Output Knowledge Created

This message produces several concrete pieces of knowledge that feed directly into the Phase 2 implementation:

  1. Two PoSpacetime parameter files exist on disk, with sizes of 57 GiB and 184 MiB respectively.
  2. The larger file is almost certainly WindowPoSt parameters, based on the circuit complexity heuristic.
  3. The smaller file is almost certainly WinningPoSt parameters.
  4. The content hashes (0377ded6... and 559e581f...) provide the exact filenames that the SRS manager will need to reference.
  5. The file modification dates (both February 17) confirm these are the current-generation parameters, consistent with the v28 naming. This knowledge directly enables the assistant to implement the CircuitId-to-filename mapping in the SRS manager. Without this investigation, the assistant would have to guess at filenames or resort to scanning the entire parameter directory at runtime—a fragile approach. With this knowledge, the mapping can be hardcoded with confidence.

The Thinking Process: Methodical Investigation

What makes this message particularly interesting is the thinking process it reveals. The assistant is engaged in a methodical, hypothesis-driven investigation. It has already:

  1. Traced the call chain (messages [msg 435]–[msg 437]) to understand how proofs flow from the API layer down to circuit construction.
  2. Found the parameter filenames (message [msg 442]) by listing the parameter directory and filtering for relevant proof types.
  3. Identified a gap in knowledge: It knows the filenames but cannot distinguish which file maps to which proof type.
  4. Formulated a hypothesis: File size will distinguish WinningPoSt from WindowPoSt.
  5. Tested the hypothesis: Runs ls -lh with a glob pattern targeting the PoSpacetime files.
  6. Interprets the results: The 57 GiB vs. 184 MiB difference confirms the hypothesis. This is classic scientific reasoning applied to software engineering. The assistant does not simply accept the ambiguity—it actively seeks disambiguating information using the tools at its disposal. The ls command is not just a file listing; it is an experiment designed to test a specific hypothesis about the relationship between parameter files and proof types.

The Broader Significance

In the larger narrative of the cuzk project, message [msg 444] represents a transition from research to implementation. The assistant has spent the preceding messages gathering knowledge: reading source files, tracing call chains, examining dependency versions, and listing directories. With the parameter file mapping resolved, it can now proceed to write the actual code for the SRS manager and the pipeline module.

The message also illustrates a recurring theme in systems programming: the gap between what the code tells you and what you need to know. The CircuitId enum tells you that different proof types exist, but it doesn't tell you which file on disk corresponds to which type. The parameter file naming convention tells you the proof family, but it doesn't distinguish between WinningPoSt and WindowPoSt. Bridging these gaps requires detective work—forming hypotheses, gathering evidence, and reasoning about the system's structure.

This is the kind of work that rarely appears in final documentation or code comments, but it is essential to building robust systems. The assistant's methodical approach—trace the code, list the files, compare the sizes, form the mapping—is a model for how to navigate complex, poorly-documented codebases. And the 57-gigabyte clue, sitting quietly on disk, holds the answer.