The Verification Grep: Confirming Memory Reservation Wiring in cuzk's Engine Refactoring

In the midst of a sweeping refactoring of the cuzk GPU proving engine's memory management system, the assistant issues a simple but critical verification command. The message, reproduced in full, is:

[grep] reservation: 
Found 4 matches
/tmp/czk/extern/cuzk/cuzk-core/src/engine.rs:
  Line 736:     pub reservation: Option<crate::memory::MemoryReservation>,
  Line 1547:                                             reservation: Some(reservation),
  Line 1828:                                             reservation: Some(reservation),
  Line 2154:                             reservation: None, // set below after unwrapping

At first glance, this appears to be a routine grep — a developer checking that a field name appears in the expected locations. But in the context of the broader refactoring effort, this single command represents a pivotal moment of verification, the culmination of dozens of surgical edits across one of the most critical files in the cuzk proving engine. To understand why this grep matters, one must understand the architecture it is validating.

The Architecture of Memory-Aware Admission Control

The cuzk engine is responsible for orchestrating GPU-based proof generation for Filecoin's proving subsystem. It handles multiple proof types — PoRep (Proof-of-Replication), WindowPoSt, SnapDeals — each with different memory footprints and parallelism characteristics. Prior to this refactoring, the engine used a static concurrency limit (partition_workers) and a semaphore-based admission control system. This approach was fragile: it could not adapt to varying proof sizes, did not account for GPU memory pressure, and had no mechanism for evicting cached data (SRS parameters or Pre-Compiled Constraint Evaluators, PCEs) when memory ran low.

The new architecture replaces this with a unified memory budget system. A MemoryBudget tracks total available GPU memory, a MemoryReservation represents an acquired slice of that budget, and an eviction-capable SrsManager and PceCache ensure that cached proving artifacts are loaded on demand and evicted when memory is scarce. At the heart of this design is the SynthesizedJob struct, which carries a synthesized proof through the GPU pipeline. The addition of a reservation: Option<MemoryReservation> field to this struct is what enables the two-phase memory release pattern that is the linchpin of the entire design.

Why This Message Was Written

The assistant had just completed a long sequence of edits to engine.rs — at least fifteen separate edit operations spanning hundreds of lines. These edits touched every major code path: the dispatcher loop, the PoRep per-partition dispatch, the SnapDeals per-partition dispatch, the monolithic synthesis path, the GPU worker loop, the synchronous fallback, and the error handling in every branch. Each edit introduced new parameters (&MemoryBudget, &PceCache), replaced old API calls (pipeline::get_pce() → pce_cache.get()), and — most importantly — added reservation fields to every SynthesizedJob construction site.

But the assistant was not done after making these edits. The pattern throughout this session is one of disciplined, methodical verification. After every batch of changes, the assistant runs grep commands to confirm that the new APIs are used consistently and that no old references remain. Earlier, the assistant verified that all five ensure_loaded() calls had the new parameter ([msg 2216]). Then it checked that all four SynthesizedJob construction sites existed ([msg 2217]). Now, in this message, it performs the final check: confirming that all four sites actually include the reservation field.

This is not mere pedantry. In a codebase of this complexity — where a missing field could cause a compilation error or, worse, a silent semantic bug where a job proceeds without a reservation, leaking memory — this verification is essential. The assistant is treating the compiler's type system as a safety net, and the grep as a pre-flight check before that net is engaged.

What the Four Matches Reveal

The four matches tell a story about the architecture and the three distinct code paths that produce SynthesizedJob instances.

Line 736 — The struct definition. This is the declaration of the reservation field itself, typed as Option<crate::memory::MemoryReservation>. The use of Option is deliberate: not every SynthesizedJob will have a reservation. In particular, the monolithic synthesis path (line 2154) initially sets it to None and fills it in later, after the synthesis result is unwrapped. The Option type makes this pattern explicit and safe.

Line 1547 — The PoRep per-partition dispatch path. Here, reservation is set to Some(reservation), where reservation was acquired via budget.acquire() before entering the spawn_blocking context. This is the path for single-sector PoRep C2 proofs, which are now dispatched through the budget-aware partition pipeline instead of the old semaphore-gated path. The reservation covers the working memory needed for synthesis and GPU proving of that partition.

Line 1828 — The SnapDeals per-partition dispatch path. Identical in pattern to the PoRep path, this site handles SnapDeals proofs. SnapDeals has 16 partitions of approximately 81 million constraints each, making its memory footprint particularly large. The reservation here is critical to prevent the engine from overcommitting GPU memory when multiple SnapDeals partitions are in flight.

Line 2154 — The monolithic synthesis path. This is the most interesting case. The reservation is set to None with a comment: "set below after unwrapping." This reflects a different control flow: in the monolithic path, the reservation is acquired before spawn_blocking (in async context), but the SynthesizedJob is constructed inside the blocking closure after synthesis completes. The reservation cannot be moved into the closure in the same expression because it is needed for error handling before the closure. Instead, the pattern is: acquire reservation, enter spawn_blocking, synthesize, construct SynthesizedJob with reservation: None, return it, unwrap the result, and then attach the reservation. This is a pragmatic concession to Rust's ownership model that still preserves the safety invariant.

The Two-Phase Release Pattern

The reservation field exists to enable the two-phase GPU memory release that is the crown jewel of this refactoring. When a SynthesizedJob reaches the GPU worker loop, the engine calls gpu_prove_start() to begin GPU computation. At this point, the a/b/c portion of the working memory (the buffers used during synthesis) can be released. The reservation is partially released after gpu_prove_start() returns, freeing that portion back to the budget. The remaining reservation — covering the GPU-side memory still in use — is then moved into a finalizer task that runs after gpu_prove_finish() completes, at which point it is fully dropped.

This two-phase approach is a significant improvement over the old design, where all memory was held until the entire proof was complete. It allows the engine to admit new work sooner, increasing throughput while staying within the memory budget.

Assumptions and Design Decisions

The verification grep validates several implicit assumptions. First, that every SynthesizedJob construction site has been found and updated — the assistant assumes that the four sites identified by the earlier grep are exhaustive. Second, that the Option<MemoryReservation> type is the correct abstraction — it assumes that the flexibility of None is needed (as in the monolithic path) and that the cost of the Option wrapper (a small memory overhead) is acceptable. Third, that the reservation should be attached to the job rather than tracked separately — this couples the memory lifecycle to the job lifecycle, which is a design choice that prioritizes clarity over flexibility.

One potential mistake is that the monolithic path's pattern of setting reservation: None and then filling it in after unwrapping creates a window where the job exists without a reservation. If an error occurs between construction and the assignment of the reservation, the job could be processed without proper memory tracking. However, the code structure ensures this window is narrow and occurs entirely within the same function scope, and the reservation is always assigned before the job is sent to the GPU worker channel.

Input and Output Knowledge

To understand this message, one must know: the structure of SynthesizedJob and its role in the pipeline; the concept of memory reservations and the budget system; the three code paths that produce synthesized jobs (PoRep partition, SnapDeals partition, monolithic); and the two-phase release pattern that the reservation enables. One must also be familiar with the assistant's methodical verification style — the pattern of grepping after every batch of edits to confirm consistency.

The knowledge produced by this message is a confirmation of correctness. The four matches tell the reader that all construction sites are wired correctly, that the struct definition is in place, and that the monolithic path's special handling (the None + set-later pattern) is intentional and documented with a comment. For anyone reviewing this refactoring, this grep output is a compact assurance that the reservation plumbing is complete.

Conclusion

A single grep command, four lines of output. But within those lines lies the validation of a complex architectural change — the replacement of a static, fragile admission control system with a dynamic, memory-aware one. The assistant's disciplined verification, checking each new field and each call site after every batch of edits, reflects a deep understanding of the codebase and a commitment to correctness. In the high-stakes world of GPU proving for Filecoin, where memory pressure can cause crashes and silent corruption, this kind of methodical validation is not optional — it is essential. The grep for reservation: is a small but telling artifact of that engineering discipline.