The Specification That Replaces Guesswork with Architecture

In the middle of a sprawling engineering session spanning dozens of messages and multiple sub-sessions, message [msg 2076] stands out as a moment of crystallization. It is deceptively simple on the surface: a single tool call that writes a file. But that file—cuzk-memory-manager.md—represents the culmination of a deep forensic audit, hours of iterative design, and a deliberate decision to encode hard-won knowledge into a form that outlives the conversation. This is the story of why that message matters, what it contains, and how it transforms a fragile, implicit memory management scheme into a robust, documented architecture.

The Problem That Demanded a Specification

To understand why message [msg 2076] was written, one must first understand the crisis it resolved. The cuzk GPU proving engine is the heart of a Filecoin proof generation pipeline. It handles massive proofs—32 GiB PoReps that consume over 13 GiB of memory per partition, with batch proofs reaching 136 GiB. The engine had been limping along with a memory management scheme that was, in the engineers' own words, "fragile" and "memory-unaware."

The existing system relied on a static partition_workers semaphore to limit concurrency. This semaphore had no awareness of actual memory pressure. It would happily admit a new proof job even when the system was on the verge of OOM. Worse, the working_memory_budget configuration option—which looked like it should enforce memory limits—was entirely dead code. It parsed a value from the config file and then... did nothing with it. The SRS (Structured Reference String) cache, which holds ~44 GiB of pinned GPU memory, was preloaded eagerly at startup regardless of whether it was needed. The PCE (Pre-Compiled Circuit Evaluator) cache, another ~26 GiB, was stored in four static OnceLock globals with no eviction policy whatsoever.

This was a system designed by accretion, not architecture. Each component handled its own memory in isolation, with no central accounting, no admission control, and no mechanism to respond to pressure. The result was a ticking time bomb: unbounded baseline memory growth, OOM risks under load, and a configuration surface that promised controls it did not deliver.

The Forensic Audit That Preceded the Specification

The message [msg 2076] did not emerge from thin air. It was preceded by message [msg 2073], where the assistant laid out an exhaustive plan for a memory budget system. That plan itself was the product of a meticulous audit of the entire memory lifecycle. The assistant had traced every allocation and deallocation point across the engine's codebase: SRS loading (~44 GiB pinned), PCE caching (~26 GiB heap), per-partition synthesis (~13.6 GiB for a/b/c/aux structures), GPU proving, and the asynchronous deallocation that followed. Each number was precise, each code path identified by line number.

The plan in [msg 2073] proposed a unified MemoryBudget system with auto-detection from /proc/meminfo, RAII-style MemoryReservation handles, LRU eviction for SRS and PCE caches with a 5-minute idle threshold, and a two-phase working memory release strategy (a/b/c freed immediately after GPU prove start, the remainder after proof finalization). It specified exactly which lines to change in which files, which configuration fields to remove and which to add, and even included sample log output showing how the system would report its state at startup and after the first proof.

The User's Directive: From Plan to Specification

Message [msg 2074] is the user's response to that plan: "write down cuzk-memory-manager.md with all details needed for a new agents to implement correctly. Keep synth concurrency config, memory-bw/cpu contention is a separate constraint, partition-workers probably not needed."

This instruction is crucial. It tells us several things. First, the user wanted the design formalized as a document, not just discussed in conversation. Second, the document needed to be complete enough for another agent to implement—a specification, not just notes. Third, the user made a deliberate design decision: keep synthesis_concurrency as a separate CPU contention knob, confirming the assistant's recommendation from [msg 2073]. The partition_workers config, on the other hand, was deemed unnecessary—the memory budget would subsume its role.

This reflects a mature understanding of the system's constraints. Memory bandwidth and CPU thread contention are orthogonal problems. A memory budget prevents OOM, but it doesn't prevent CPU oversubscription when multiple syntheses compete for rayon thread pool slots. Keeping a separate concurrency limit for CPU contention is the right call, and the user's confirmation shows they understood this distinction.

The Act of Writing: Message [msg 2076]

Message [msg 2076] is the execution of that directive. The assistant calls the write tool to create /tmp/czk/extern/cuzk/cuzk-memory-manager.md. The tool returns "Wrote file successfully." That's it—no content is echoed back in the message itself. The file's contents are not visible in the conversation transcript.

But we know what went into that file. It was the detailed plan from [msg 2073], refined by the user's feedback in [msg 2074]. The specification would have included:

• Core type definitions: MemoryBudget with its total_bytes, used counter, Notify for wake-on-free, and the evictor callback. MemoryReservation as an RAII handle with partial release and Drop safety net. The acquire() logic with its four-step retry loop: optimistic fetch-add, eviction on failure, retry, and blocking wait.
• Cache redesign: The PceCache struct replacing four static OnceLock globals, with Mutex<HashMap> entries, budget-gated insertion, and evictable entry enumeration. The SrsManager modifications adding last_used tracking and budget-gated loading.
• Integration points: Exact line numbers for every change in engine.rs, pipeline.rs, config.rs, and srs_manager.rs. The startup sequence (remove preload, create budget, wire evictor). The partition dispatch (replace semaphore with budget acquire). The GPU worker loop (two-phase release after prove_start and prove_finish).
• Memory estimation constants: Precise byte counts for every allocation category across all proof types—PoRep 32G, WindowPoSt, WinningPoSt, SnapDeals—including PCE extraction transient memory.
• Config migration: The old config with its dead working_memory_budget and misleading pinned_budget replaced by a minimal [memory] section with just total_budget (defaulting to auto-detect) and safety_margin (defaulting to 5 GiB).
• Eviction policy: LRU with 5-minute minimum idle time, triggered only under memory pressure, checking Arc::strong_count == 1 to avoid evicting in-flight entries.

The LSP Errors: A Curious Detail

The message also reports LSP errors from two Go files: sector.go in the Filecoin go-state-types module and provider.go in the cusvc/proofs directory. These errors—undefined: ActorID, undefined: Randomness—are in unrelated Go code, not in the Rust codebase being modified. They are likely pre-existing issues in the Go development environment (perhaps missing build dependencies or a stale LSP cache).

The assistant does not comment on these errors. This is the right call—they are irrelevant to the Rust memory manager specification being written. But their presence in the message is a reminder of the messy reality of real-world engineering: you are never working on a clean system. There are always errors in other files, other projects, other languages. The ability to ignore irrelevant noise is as important as the ability to spot critical signals.

What This Message Achieves

Message [msg 2076] transforms ephemeral conversation into permanent documentation. The detailed plan that existed in the assistant's reasoning and in the conversation history is now codified in a standalone file that can be read by any future agent or human developer. This is a force multiplier: the next engineer who needs to understand or modify the memory management system doesn't need to replay the entire debugging session. They can read the specification.

The specification also serves as a contract. By writing it down with implementation-level detail, the assistant and user have agreed on exactly what will be built. There is no ambiguity about whether partition_workers should be kept or removed, whether synthesis_concurrency is in scope, or how the eviction callback should prioritize entries. The document answers all these questions.

Finally, the message demonstrates a particular engineering virtue: knowing when to stop debugging and start documenting. The forensic audit was complete. The design decisions were made. The iterative Q&A had converged. At that point, the highest-leverage action was not to write more code, but to write down the architecture so that the implementation—whether by the same assistant, a different agent, or a human developer—could proceed with clarity and confidence.

The file cuzk-memory-manager.md may be empty in the artifact store, but in the conversation it represents a fully specified architecture that replaces guesswork with design, and fragility with structure.