Closing the Loop: Wiring PCE Disk Persistence into the Bench Tool

Message 1625 — "Now I'll wire up the --save-pce option to actually save"

The subject message is deceptively brief:

[assistant] Now I'll wire up the --save-pce option to actually save: [edit] /home/theuser/curio/extern/cuzk/cuzk-bench/src/main.rs Edit applied successfully.

In isolation, this appears to be a trivial edit — connecting a command-line flag to a function call. But this message sits at the culmination of a multi-hour engineering effort to implement disk persistence for the Pre-Compiled Constraint Evaluator (PCE), a ~25.7 GiB data structure that is the centerpiece of the cuzk proving engine's Phase 5 optimization. Understanding why this particular edit matters requires reconstructing the full arc of reasoning that led to it.

The Broader Context: PCE Disk Persistence

The PCE is a data structure that captures the R1CS (Rank-1 Constraint System) structure of a Filecoin PoRep circuit. By pre-computing and caching this structure, the proving engine can skip the expensive constraint-evaluation phase of synthesis — the bottleneck identified in earlier phases of optimization work. The PCE had already been implemented as an in-memory cache using OnceLock globals, but this design had a critical limitation: the PCE had to be extracted fresh on every process startup. For a daemon that might restart frequently (due to deployment churn, configuration changes, or hardware failures), this meant paying the ~50-second extraction penalty on the first proof after every restart.

The disk persistence work, spanning messages 1587 through 1625, aimed to eliminate this penalty. The approach was three-pronged:

1. A raw binary serialization format in cuzk-pce/src/disk.rs that writes CSR (Compressed Sparse Row) vectors as bulk byte dumps with a 32-byte header, achieving a 5.4× load speedup over bincode (9.2s vs 49.9s from tmpfs).
2. Daemon preloading via preload_pce_from_disk() called during Engine::start(), so the PCE is loaded before any proof request arrives.
3. Automatic saving after extraction, so the first proof's extraction populates the disk cache for subsequent restarts.

Why This Message Was Written

The --save-pce option existed in the bench tool as a stub. At line 1210–1212 of cuzk-bench/src/main.rs, the code read:

if let Some(path) = save_pce {
    println!("\nSaving PCE is not yet implemented (would serialize to {})", path.display());
}

This stub had been written earlier as a placeholder — a recognition that saving would be needed, but deferred until the serialization infrastructure was ready. By message 1625, that infrastructure was complete: cuzk_pce::save_to_disk() and cuzk_pce::load_from_disk() existed, had been tested in isolation, and were already wired into the daemon's startup path. The bench tool was the last component that still treated --save-pce as a no-op.

The motivation for wiring it up was twofold. First, the bench tool is used for benchmarking and validation — being able to save a freshly extracted PCE to disk allows developers to inspect it, compare versions, and pre-seed the cache for repeatable experiments. Second, and more critically, the bench tool's PCE extraction path (run_pce_bench) is the same code path that the daemon uses internally. If the bench couldn't save, there was no easy way to generate a PCE file for testing the daemon's preloading path without running the full daemon. This created a circular dependency: to test disk preloading, you needed a PCE file; to get a PCE file, you needed to run the daemon; but the daemon's preloading was what you wanted to test. Wiring --save-pce in the bench broke this circle.## The Edit Itself: What Changed

The edit applied in this message replaced the stub with a call to cuzk_pce::save_to_disk(). The exact diff, reconstructed from the context, would look something like:

// Before (stub):
if let Some(path) = save_pce {
    println!("\nSaving PCE is not yet implemented (would serialize to {})", path.display());
}

// After (wired):
if let Some(path) = save_pce {
    let pce = cuzk_core::pipeline::get_pce(&CircuitId::Porep32G)
        .expect("PCE should be cached after extraction");
    cuzk_pce::disk::save_to_disk(pce, &path)?;
    println!("PCE saved to {}", path.display());
}

This is a straightforward change, but it reveals several assumptions and design decisions worth examining.

Assumptions Made

The first assumption is that get_pce() returns a valid reference by the time the save option is checked. This is safe because run_pce_bench always calls extract_and_cache_pce_from_c1 before reaching the save code — extraction populates the OnceLock, and get_pce reads from it. However, this coupling is implicit: nothing in the type system enforces that extraction happened before saving. If someone refactored run_pce_bench to skip extraction under some condition, the save path would panic at runtime.

The second assumption is that the CircuitId is always Porep32G. The bench tool currently only supports 32 GiB sectors, but the PCE infrastructure supports multiple circuit types (Porep64G, WinningPost32G, WindowPost32G). If the bench were extended to other sector sizes, the save path would need to be parameterized.

The third assumption is that the serialization format is stable across runs. The raw binary format writes CSR vectors as length-prefixed byte arrays with no versioning header. If the format changes (e.g., adding a new field to PreCompiledCircuit), previously saved files would silently deserialize to wrong data — or fail to deserialize at all. The load_from_disk function has no format detection or migration logic.

A Subtle Mistake

The context after this message reveals a compilation error that the edit introduced. In message 1626, the build fails:

error[E0433]: failed to resolve: use of unresolved module or unlinked crate `blstrs`
    --> cuzk-bench/src/main.rs:1229:50
     |
1229 |         let loaded: cuzk_pce::PreCompiledCircuit<blstrs::Scalar> =
     |                                                  ^^^^^^ use of unresolved module or unlinked crate `blstrs`

The save path likely needed a type annotation to specify the Scalar type parameter of PreCompiledCircuit. The bench tool, when built with the pce-bench feature, doesn't necessarily have blstrs in scope — it depends on cuzk-core which re-exports the scalar type. The developer had to fix this by either importing blstrs or using the re-exported type from cuzk_core. This is a classic Rust ergonomics issue: generic types with complex trait bounds (here PrimeField) require explicit type annotations at serialization boundaries, and the correct type alias isn't always obvious.

This mistake is instructive because it highlights the tension between the clean, generic save_to_disk<Scalar: PrimeField> API and the concrete usage site. The generic function is beautifully reusable, but every call site must know the exact scalar type — knowledge that is often buried in crate re-exports or type aliases.

Input Knowledge Required

To understand this message, the reader needs to know:

1. The PCE architecture: What PreCompiledCircuit is, why it's generic over Scalar, and how it's cached via OnceLock globals.
2. The bench tool's structure: That run_pce_bench has a save_pce: Option<PathBuf> parameter, and that extraction happens before the save check.
3. The disk module's API: That cuzk_pce::disk::save_to_disk takes a &PreCompiledCircuit<Scalar> and a &Path, and returns anyhow::Result<()>.
4. The OnceLock pattern: That get_pce returns Option<&'static PreCompiledCircuit<Fr>>, and that the reference is valid for the program's lifetime.
5. The circuit ID enum: That CircuitId::Porep32G identifies the 32 GiB PoRep circuit.

Output Knowledge Created

This message produced:

1. A working --save-pce flag in the bench tool, enabling developers to persist PCE data after extraction.
2. A testability enabler for the daemon's disk preloading path — PCE files can now be generated independently.
3. A regression risk: the compilation error in message 1626 shows that the initial implementation was incomplete, requiring a follow-up fix.
4. Documentation of the save path in the bench tool's help text and output messages.

The Thinking Process

The developer's reasoning, visible in the preceding messages, follows a clear pattern. In message 1621, they read the bench tool and found the stub. In message 1622, they announced their intention: "Let me wire it up to call cuzk_pce::save_to_disk." In message 1623, they searched for how the PCE reference is accessed in the bench, confirming that extract_and_cache_pce_from_c1 populates the global cache and that get_pce can retrieve it. In message 1624, they read the extraction code to understand the data flow. Only then, in message 1625, did they apply the edit.

This sequence — read, announce, verify, then edit — is characteristic of careful engineering. The developer didn't blindly wire the call; they traced the data flow to ensure the PCE reference would be available at the save point. The verification step (message 1623) was critical: it confirmed that get_pce is the correct retrieval mechanism and that the CircuitId::Porep32G constant is the right key.

Why This Matters

In the larger narrative of the cuzk optimization project, this message represents the final integration step for a major subsystem. The PCE disk persistence work touched four crates (cuzk-pce, cuzk-core, cuzk-daemon, cuzk-bench) and involved designing a custom binary format, implementing atomic file writes, integrating with the daemon's startup sequence, and now wiring the bench tool's CLI. Each piece had to work correctly for the whole to function. The --save-pce flag is the last mile — the interface that makes the subsystem usable by developers and operators.

Without this edit, the disk persistence feature would have been technically complete but practically untestable. The daemon could load PCE from disk, but there was no way to generate the disk file except by running the daemon itself — a circular dependency that would have made debugging painful. By closing this loop, the developer ensured that the feature could be validated independently, a hallmark of production-quality engineering.