The Quiet Removal: How a Single Grep Command Signaled an Architectural Shift in cuzk's Memory Manager

In the middle of a sweeping refactor to introduce a unified memory manager for the cuzk GPU proving engine, the assistant issued a message that appears, at first glance, utterly mundane:

Now find and replace preload_pce_from_disk: [grep] ^pub fn preload_pce_from_disk Found 1 matches /tmp/czk/extern/cuzk/cuzk-core/src/pipeline.rs: Line 584: pub fn preload_pce_from_disk(param_cache: &std::path::Path) -> usize {

A grep command. A single line of output. A function signature. On its surface, this is one of the most forgettable moments in any coding session—the kind of mechanical step that developers perform dozens of times a day without a second thought. But in the context of the larger transformation underway, this message marks a precise inflection point: the moment when an old architectural paradigm is located, targeted, and prepared for removal.

The Context: A Memory Manager Born from Production Pain

To understand why this grep matters, one must understand the problem that brought it into existence. The cuzk engine is a GPU-accelerated proving system for Filecoin's proof-of-replication (PoRep) and proof-of-spacetime (PoSt) circuits. It is a performance-critical component that must juggle multiple expensive resources: Structured Reference Strings (SRS) that can consume gigabytes of GPU memory, Pre-Compiled Constraint Evaluators (PCE) that represent synthesized circuits, and the working memory needed for actual proof generation.

The old system had grown organically and was showing its age. It relied on four static OnceLock<PreCompiledCircuit<Fr>> globals in pipeline.rs—one for each proof type (PoRep, SnapDeals, WindowPoSt, WinningPoSt). These were loaded once at startup and never released. A configuration field called working_memory_budget existed in the code but was effectively dead, never wired into actual enforcement. The partition_workers setting attempted to limit concurrency but did so through a fragile static cap rather than any awareness of actual memory pressure. The system had no eviction mechanism, no unified budget tracking, and no way to gracefully handle the memory demands of concurrent proof generation.

This was the state of affairs when the assistant began implementing the specification laid out in cuzk-memory-manager.md. The goal was nothing less than a complete replacement of the memory architecture: a unified MemoryBudget with admission control, an LRU-evictable PceCache to replace the static globals, a budget-aware SrsManager that could release SRS entries under pressure, and a two-phase working memory release protocol for partition synthesis.

The Message: A Surgical Strike on an Obsolete Function

By the time the assistant issued message [msg 2111], four of the seven planned implementation steps were already complete. The memory.rs module had been created with MemoryBudget, MemoryReservation, detect_system_memory(), and estimation constants for every proof type. The config.rs had been rewritten to replace dead configuration fields with the new unified budget model. The srs_manager.rs had been made budget-aware with last_used tracking and eviction support. And in pipeline.rs, the four static OnceLock globals had already been replaced by the new PceCache struct.

But one loose end remained: the preload_pce_from_disk function.

This function embodied the old philosophy. It was called at daemon startup to eagerly load PCE files from the parameter cache directory for all known circuit types, returning a count of how many were successfully loaded. It was a blunt instrument—load everything, hope it fits, never think about it again. In the new architecture, this approach was not just obsolete but actively harmful. The new PceCache was designed for on-demand loading: PCEs would be extracted and cached only when first needed, and could be evicted under memory pressure. Preloading at startup would bypass the budget system entirely, potentially consuming memory that should be reserved for active proof jobs.

The grep command in message [msg 2111] was the first step of a surgical removal. The assistant needed to locate the exact position of the function before applying an edit to delete it. The ^pub fn preload_pce_from_disk regex was carefully chosen to match only the function declaration itself, not any call sites or documentation comments. The result confirmed the function lived at line 584 of pipeline.rs.

The Reasoning Behind the Removal

The decision to remove preload_pce_from_disk rather than adapt it was not made lightly. The assistant's earlier reasoning, visible in the planning stages of this chunk ([chunk 15.0]), reveals a careful consideration of trade-offs. The function could have been modified to use PceCache internally, but that would have preserved the preload-at-startup semantic, which was fundamentally at odds with the new memory-budget approach. Preloading assumes that all PCEs will be needed and that memory is plentiful—assumptions that had already proven false in production, where the old system's lack of memory pressure handling had caused crashes and instability.

The assistant's thinking, documented in earlier messages, shows an awareness of the deeper architectural shift: "Now update load_pce_from_disk and preload_pce_from_disk functions. They need to be adapted to work with PceCache. Let me update load_pce_from_disk to become a standalone function that returns the PCE (for backward compat with bench tool), and modify preload_pce_from_disk to be removed since loading is now on-demand" ([msg 2109]). This distinction is important: load_pce_from_disk was preserved as a utility for the benchmarking tool, while preload_pce_from_disk was marked for deletion because its startup-time-eager-loading semantic had no place in the new system.## The Assumptions Underpinning the Decision

The removal of preload_pce_from_disk rested on several assumptions, some explicit and some implicit. The most critical was that on-demand loading with eviction would provide acceptable performance. This assumption was grounded in the observation that PCE extraction happens only once per circuit type—after the first proof of each type completes, the PCE is cached and reused. The cost of the first proof being slightly slower (because extraction happens synchronously during that first proof) was deemed acceptable compared to the risk of preloading consuming memory that could trigger OOM conditions.

Another assumption was that the benchmarking tool, which directly called load_pce_from_disk and get_pce with a circuit ID, could continue to work with the adapted standalone version of load_pce_from_disk. The assistant preserved backward compatibility for this use case, recognizing that the bench tool operated in a controlled environment where preloading was acceptable.

A more subtle assumption concerned the thread safety of the transition. The old OnceLock statics provided a simple, safe initialization pattern: exactly one thread could set the value, and all subsequent reads would see the initialized data. The new PceCache used a Mutex<HashMap<CircuitId, Arc<PreCompiledCircuit<Fr>>>> internally, which introduced different concurrency characteristics. The assistant assumed that the overhead of mutex locking on cache lookups would be negligible compared to the GPU proving work being performed—a reasonable assumption given that cache hits are fast hash map lookups while GPU synthesis takes seconds or minutes.

Input Knowledge Required

To understand the significance of this message, one must be familiar with several pieces of context that are not visible in the grep output itself. First, the overall architecture of the cuzk engine: it is a GPU proving system that processes Filecoin proof types (PoRep, SnapDeals, WindowPoSt, WinningPoSt), each with different circuit structures and memory footprints. Second, the old memory management approach: four static OnceLock globals, a dead working_memory_budget config field, and a partition_workers concurrency limit that approximated memory control without actually measuring memory. Third, the specification document cuzk-memory-manager.md, which laid out the unified budget architecture that this implementation follows.

One must also understand the Rust-specific concepts at play: OnceLock (a synchronization primitive for one-time initialization), Arc (atomic reference counting for shared ownership), feature gating with #[cfg(feature = "cuda-supraseal")], and the distinction between async and blocking contexts that complicated the budget acquisition design. The assistant's earlier reasoning wrestled extensively with this async/blocking tension, particularly around how SrsManager::ensure_loaded could acquire budget from a blocking context without deadlocking the tokio runtime.

Output Knowledge Created

This message, though brief, contributed to the creation of significant output knowledge. By locating and flagging preload_pce_from_disk for removal, it established that the function was a single point of change—one function, one file, one edit. The subsequent edit (message [msg 2113]) replaced the function body with a comment: "// preload_pce_from_disk removed — PCE loading is now on-demand via PceCache." This comment itself became a piece of output knowledge, documenting for future readers why the function disappeared and what replaced it.

More broadly, the entire chunk ([chunk 15.0]) created a comprehensive new memory management subsystem. The PceCache struct, the updated extract_and_cache_pce_from_* functions that now accept &PceCache, the synthesize_auto function with its optional &PceCache parameter updated across 9 call sites—all of this constitutes new knowledge about how the system manages its most expensive resource.

The Thinking Process: From Spec to Implementation

The assistant's reasoning, visible across the messages leading up to this one, reveals a careful, methodical approach to a complex refactoring. The thinking process began with a thorough reading of all source files to understand the current state (message [msg 2090] shows extensive reasoning about the design). The assistant considered multiple design alternatives: whether to place PceCache in memory.rs or pipeline.rs, whether to use tokio::sync::RwLock or std::sync::RwLock for the evictor callback, how to handle the async/blocking boundary for SRS loading, and whether to use MemoryReservation for permanent allocations or add raw budget manipulation methods.

The decision to remove preload_pce_from_disk specifically came after the assistant had already replaced the static OnceLock globals with PceCache (message [msg 2108]) and adapted load_pce_from_disk to return a standalone PCE (message [msg 2109]). At that point, preload_pce_from_disk was the last vestige of the old eager-loading approach. The grep command in message [msg 2111] was the reconnaissance step before the deletion—confirming the function's exact location so the edit could be precise.

Mistakes and Correctness Considerations

Was the removal of preload_pce_from_disk correct? In the context of the new architecture, yes. But the decision carried risks. If the on-demand loading path had a bug that caused PCE extraction to fail silently, the system would fall back to synthesis without PCE for every proof, dramatically reducing performance. The old preloading approach, for all its simplicity, at least guaranteed that PCEs were available after startup.

The assistant mitigated this risk by preserving load_pce_from_disk as a standalone utility. If on-demand loading failed, the benchmarking tool and diagnostic paths could still load PCEs directly. Additionally, the PceCache was designed with a fallback: if a PCE was not in cache and could not be loaded from disk, the system would synthesize without it rather than crash.

Another potential issue was the interaction between PCE eviction and the GPU worker pipeline. If a PCE was evicted while a GPU worker was actively using it, the worker's Arc<PreCompiledCircuit<Fr>> reference would keep the data alive, but the cache would lose track of it. The next proof of the same type would need to re-extract or re-load the PCE. The assistant's design handled this correctly through reference counting, but the performance implications of frequent eviction-and-reload cycles were not fully explored.

Conclusion

A grep command is rarely the subject of deep analysis. But in the context of a carefully planned architectural transformation, even a mechanical search-and-locate operation carries meaning. Message [msg 2111] represents the precise moment when the old paradigm of eager preloading was located, identified as obsolete, and marked for excision. It is a reminder that architectural change happens not only in grand design documents and sweeping rewrites, but also in the quiet removal of functions that no longer fit the world they helped create.