The Quiet Deletion: Removing preload_pce_from_disk in the cuzk Memory Manager

Message: [edit] /tmp/czk/extern/cuzk/cuzk-core/src/pipeline.rs — "Edit applied successfully."

At first glance, message [msg 2113] appears to be the most mundane event in a software engineering session: a tool call confirmation reporting that an edit was applied successfully. There is no reasoning, no explanation, no thinking block — just a flat, mechanical acknowledgment. Yet this single line represents a critical inflection point in the implementation of a comprehensive memory management system for the cuzk GPU proving engine. To understand why, one must trace the chain of reasoning that led to this moment, the architectural philosophy it embodies, and the assumptions it discards.

The Context: A Memory Crisis in the GPU Proving Engine

The cuzk daemon is a high-performance GPU-based zero-knowledge proof generator for the Filecoin network. It handles multiple proof types — PoRep (Proof-of-Replication), WindowPoSt, WinningPoSt, and SnapDeals — each requiring enormous amounts of GPU memory. A single 32 GiB PoRep proof, for example, requires approximately 70 GiB of baseline resident memory (SRS parameters at ~44 GiB plus Pre-Compiled Circuit Evaluator structures at ~26 GiB), plus an additional ~13.6 GiB per partition for working memory during proof generation.

The existing memory management was fragile. The working_memory_budget configuration field was dead code — never checked anywhere. The pinned_budget field was advisory only, logging warnings but never preventing oversubscription. The partition_workers setting was a static count, not memory-aware, meaning wrong values could cause out-of-memory crashes or GPU starvation. Most critically, the Pre-Compiled Circuit Evaluator (PCE) caches were stored in four static OnceLock<PreCompiledCircuit<Fr>> globals — write-once, never evicted, accumulating across all circuit types. If all four proof types were used, the baseline memory footprint could exceed 200 GiB with no eviction mechanism.

The solution was a unified, budget-based memory manager, specified in /tmp/czk/extern/cuzk/cuzk-memory-manager.md. The core idea: replace static concurrency limits with a single byte-level budget auto-detected from system RAM, with on-demand loading and LRU eviction of SRS and PCE structures under memory pressure.

The Subject Message: What Actually Happened

Message [msg 2113] is the edit that removed the preload_pce_from_disk function from pipeline.rs. This function, gated behind #[cfg(feature = "cuda-supraseal")], was responsible for preloading PCE caches for all known circuit types from disk at daemon startup. It would iterate over the parameter cache directory, load whatever PCE files existed, and return the count of successfully loaded circuit types. The function was called from the daemon's startup sequence to warm the cache before any proofs were processed.

The edit replaced the entire function body with a single comment: // preload_pce_from_disk removed — PCE loading is now on-demand via PceCache. This is visible in the subsequent message [msg 2114], where the assistant reads the file and the comment appears at line 579.

But this deletion was not an isolated act. It was the culmination of a multi-step transformation of pipeline.rs that began in [msg 2108], where the four static OnceLock PCE caches were replaced with a new PceCache struct. In [msg 2109], load_pce_from_disk was refactored from a method that populated the static caches into a standalone function that returns a PCE (preserving backward compatibility with the bench tool). And in [msg 2113], preload_pce_from_disk — the function that called load_pce_from_disk at startup for each circuit type — was removed entirely.

Why This Deletion Matters: The Shift from Proactive to Reactive Loading

The removal of preload_pce_from_disk embodies a fundamental design shift: moving from proactive resource acquisition to reactive, demand-driven resource management.

Under the old design, the daemon would eagerly load all PCE structures at startup, regardless of whether they would ever be used. This was simple and predictable — if a daemon was configured for PoRep proofs, it loaded the PoRep PCE at boot, and it stayed in memory forever. But this approach had two critical flaws. First, it consumed memory for circuit types that might never be requested, which was wasteful in a multi-tenant or dynamically-configured environment. Second, it had no mechanism for release — once loaded, the PCE was pinned in a OnceLock for the lifetime of the process, contributing to the accumulating baseline that could exceed 200 GiB.

The new PceCache struct, by contrast, loads PCE structures on demand — only when a proof of that type is first encountered. And crucially, it supports eviction: entries track their last_used timestamp, and under memory pressure, idle entries can be evicted to free budget for other work. The eviction policy requires entries to be idle for a configurable minimum duration (defaulting to 5 minutes), preventing thrashing while still allowing the system to reclaim memory from unused circuit types.

This shift from "load everything, never unload" to "load what you need, evict what you don't" is the heart of the memory-aware architecture. The deletion of preload_pce_from_disk is the symbolic act that completes this transition — removing the last vestige of the eager-loading paradigm.## The Assumptions Embedded in This Edit

The deletion of preload_pce_from_disk rests on several assumptions that deserve scrutiny.

First assumption: on-demand loading is fast enough. The old design preloaded PCEs at startup precisely because loading from disk is not instantaneous — reading a ~26 GiB PCE file at ~5 GB/s takes several seconds. The new design assumes that the latency of loading a PCE on first demand is acceptable, either because the daemon can queue the proof request and process it once the PCE is ready, or because the first proof of each type will be slower but subsequent proofs will benefit from the cached result. This is a reasonable trade-off for a proving engine that processes proofs asynchronously, but it does shift complexity from startup time to runtime.

Second assumption: eviction is safe. The old OnceLock design guaranteed that once a PCE was loaded, it would never be dropped. The new PceCache allows eviction under memory pressure. This assumes that the cost of re-loading an evicted PCE (another ~5 seconds of disk I/O) is less than the cost of holding it in memory indefinitely. For infrequently-used circuit types, this is almost certainly true. But it also assumes that the eviction policy — idle for 5 minutes — is a good heuristic for "not needed soon." This is a heuristic that may need tuning in production.

Third assumption: the bench tool's backward compatibility is worth preserving. The refactoring kept load_pce_from_disk as a standalone function (returning a PreCompiledCircuit<Fr>) specifically so the bench tool could still use it. This suggests an awareness that the bench tool operates outside the normal daemon lifecycle and needs direct access to PCE loading. The assumption is that the bench tool's use case is sufficiently different from the daemon's that it should not be forced into the on-demand paradigm.

The Thinking Process Visible in the Reasoning

While message [msg 2113] itself contains no reasoning — it is a bare tool call confirmation — the thinking process is visible in the sequence of edits that surround it. The assistant's approach reveals a careful, methodical strategy.

In [msg 2108], the assistant begins by replacing the four static OnceLock globals with the PceCache struct. This is the foundational change — everything else depends on it. The assistant does not attempt to modify the statics in place; instead, it introduces a new abstraction that encapsulates both storage and eviction logic.

In [msg 2109], the assistant refactors load_pce_from_disk to be a standalone function. This is a transitional step: the function still exists and still loads a PCE, but it no longer writes into a static global. Instead, it returns the loaded PCE to the caller, who can then decide whether to cache it in the PceCache or use it directly (as the bench tool does).

In [msg 2110], the assistant reads the full preload_pce_from_disk function to understand its structure before deleting it. This is a deliberate, cautious approach — the assistant wants to see exactly what it's removing, not blindly delete by line number.

In [msg 2111], the assistant uses grep to confirm the exact location of the function. This double-checking suggests an awareness that the function might have callers or references elsewhere that need updating.

Finally, in [msg 2113], the edit is applied. The assistant does not rush; it proceeds step by step, ensuring each change compiles and is consistent with the others.

Input Knowledge Required

To understand this message, one must know:

  1. The cuzk memory architecture: That PCE structures are large (~26 GiB each), that they were stored in static OnceLock globals, and that they accumulated across circuit types without eviction.
  2. The old startup flow: That preload_pce_from_disk was called at daemon startup to warm the cache, iterating over the parameter cache directory.
  3. The new PceCache design: That the replacement is a struct supporting on-demand loading, last_used tracking, and eviction under memory pressure, integrated with the unified MemoryBudget.
  4. The broader memory manager specification: That the entire system is shifting from static, proactive resource management to dynamic, budget-based management.

Output Knowledge Created

This message creates knowledge about the new architecture's boundaries:

Conclusion

Message [msg 2113] is a small edit with large implications. It marks the completion of a fundamental architectural transition in the cuzk proving engine: from eager, static resource allocation to demand-driven, evictable caching. The deletion of preload_pce_from_disk is not merely the removal of a function — it is the removal of an entire philosophy of memory management. In its place stands the PceCache, a structure that embodies the new philosophy of memory-aware, budget-constrained, eviction-supported resource management. The edit confirmation is terse, but the reasoning behind it is deep, spanning multiple files, dozens of design decisions, and a thorough understanding of GPU proving memory lifecycles.