The Architecture of Delegation: Designing the cuzk–Curio Integration Plan

Introduction

In any complex software system, the moment when investigation crystallizes into a concrete plan is a pivotal one. It marks the transition from understanding what exists to deciding what to build. Message 3366 in this opencode session captures exactly such a moment: an AI assistant, after an extensive exploration of Curio's task orchestrator and the cuzk proving daemon's codebase, produces a comprehensive four-part integration plan that will reshape how Filecoin proof generation is scheduled and executed.

This message is not merely a list of tasks. It is an architectural document that reveals deep reasoning about system boundaries, resource accounting, scheduling semantics, and the delicate art of delegating responsibility between two independent subsystems. The plan proposes to wire the cuzk remote proving daemon into Curio's Harmony task orchestrator, covering configuration, client generation, scheduler modification, and computation routing. In doing so, it fundamentally reimagines the relationship between Curio's local resource management and the external proving infrastructure.

To understand why this message matters, we must examine the context that produced it, the reasoning embedded within it, the assumptions it makes, and the architectural philosophy it embodies.

The Road to the Plan: Context and Investigation

The message at index 3366 did not emerge in a vacuum. It is the culmination of a sustained investigation spanning messages 3332 through 3365, during which the assistant systematically explored Curio's codebase to understand the integration surface between the two systems.

The investigation began with a broad question: how does Curio currently schedule and execute proof-generation tasks? The assistant ran a series of find and grep commands to locate the relevant source files. It discovered the task definitions in tasks/seal/, tasks/snap/, tasks/window/, and tasks/winning/, each of which implements the harmonytask.TaskType interface. It examined TypeDetails() methods to understand how tasks declare their resource requirements (GPU count, RAM), and CanAccept() methods to understand how the scheduler decides whether a task can run on the local machine.

A critical discovery came when the assistant inspected tasks/seal/task_porep.go and found the PoRepTask structure, which carries an *ffi.SealCalls field and an enableRemoteProofs boolean. This revealed that Curio already had a nascent concept of remote proof delegation, though it was limited and not yet connected to the cuzk daemon. The assistant also examined tasks/snap/task_prove.go and tasks/window/compute_task.go, confirming that the same pattern of FFI-backed proof generation repeated across all proof types.

Parallel to this code exploration, the assistant investigated the cuzk side of the equation. It located the protobuf definitions in extern/cuzk/cuzk-proto/proto/cuzk/v1/proving.proto and read the service definitions: SubmitProof, AwaitProof, Prove, CancelProof, and GetStatus. This gave the assistant a clear picture of the gRPC API that the daemon exposes. It also noted that no Go client library yet existed — lib/cuzk/client.go was absent, meaning the integration would need to generate protobuf stubs and build a client wrapper from scratch.

By message 3362, the assistant had identified the core architectural tension: Curio's Harmony scheduler uses local resource accounting (GPU count, RAM) to decide whether to accept tasks, but the cuzk daemon manages its own GPU resources independently. If Curio continued to count local GPU resources for tasks that would actually run on the daemon, the scheduler would either double-count resources or, worse, refuse to schedule tasks because local GPUs appeared busy when they were not being used at all. This insight became the central design challenge that the plan in message 3366 would address.

Anatomy of the Plan: Four Pillars of Integration

The plan presented in message 3366 is structured around four distinct workstreams, each addressing a specific layer of the integration. The assistant presents them in a logical order that mirrors the dependency chain: configuration first, then client infrastructure, then scheduler modification, and finally computation wiring.

Pillar 1: Configuration Additions

The plan begins at the configuration layer, proposing a new CuzkConfig struct to be added to deps/config/types.go. This struct encapsulates five fields:

Pillar 2: gRPC Client Infrastructure

The second pillar addresses the missing client library. The plan calls for:

  1. A lib/cuzk/generate.go file containing a //go:generate directive to compile the protobuf definitions into Go stubs.
  2. A lib/cuzk/client.go file wrapping the generated ProvingEngineClient with a long-lived gRPC connection supporting auto-reconnects.
  3. High-level wrappers for SubmitAndAwait, GetStatus, and CancelProof. This pillar is the most straightforward technically, but it carries significant operational implications. The decision to use a long-lived connection with auto-reconnects rather than per-request connections reflects an assumption that the daemon will be a stable, long-running process — consistent with the "Persistent Prover Daemon" architecture established earlier in the project. The auto-reconnect logic also implies that Curio should tolerate daemon restarts gracefully, which is essential for production deployments. The choice of SubmitAndAwait as a combined wrapper (rather than requiring separate SubmitProof and AwaitProof calls) is a pragmatic concession to the synchronous nature of Curio's task execution model. The protobuf definition includes both a split API (Submit + Await) and a combined API (Prove), and the plan opts for the combined form, suggesting a preference for simplicity over the latency-hiding benefits of the split API.

Pillar 3: Scheduler Modification via CanAccept and Resource Delegation

This is the most architecturally significant pillar, and the one where the assistant's reasoning is most visible. The plan identifies a fundamental conflict: Curio's Harmony scheduler uses TypeDetails() to learn a task's resource requirements and CanAccept() to decide whether those resources are available locally. If cuzk is handling the GPU work, Curio should not be reserving local GPU capacity for those tasks.

The solution is twofold:

A. Zeroing out resource costs in TypeDetails(): When the relevant cuzk enable flag is true, the GPU and RAM fields in the task's Cost are set to zero. This effectively tells the Harmony scheduler that the task requires no local resources, allowing it to be scheduled freely without regard to local GPU availability.

B. Implementing backpressure in CanAccept(): Instead of checking local resource availability, CanAccept() queries the cuzk daemon's GetStatus() endpoint to learn the current queue depth. If the pending count exceeds MaxPending, the task is rejected, signaling to the scheduler that it should wait. If there is capacity, the method returns a subset of the proposed task IDs, up to the available slots.

This design is elegant because it preserves the existing Harmony scheduling semantics while substituting the resource oracle. The scheduler continues to call CanAccept() exactly as before, but the implementation now consults a remote queue rather than a local resource monitor. The abstraction is clean: from the scheduler's perspective, cuzk is simply another resource provider with a finite capacity.

However, the plan reveals an important subtlety in the backpressure logic. The code snippet shows:

return ids[:min(len(ids), config.Cuzk.MaxPending - pending)], nil

This returns a slice of the input ids array, accepting only as many tasks as there are available slots in the daemon's queue. This is correct behavior for the Harmony API, which expects CanAccept to return the subset of proposed tasks that the task type is willing to accept. But it also means that the scheduler might propose more tasks than the daemon can handle, and CanAccept will silently drop the excess. The scheduler will then propose those tasks again in a future round, creating a natural retry loop.

Pillar 4: Wiring the Computations

The final pillar addresses the Do() methods — the actual execution paths. For each proof type, the plan specifies how to serialize the proof inputs and which ProofKind to use in the gRPC request:

The Reasoning Process: What the Thinking Reveals

The "Agent Reasoning" section at the top of message 3366 provides a window into the assistant's cognitive process. It reveals three distinct trains of thought:

First, the assistant is considering the feasibility of SnapProve — whether the existing p.sc.SnapProve function can be adapted for remote execution. This is a concrete technical question that builds on the earlier investigation of tasks/snap/task_prove.go. The assistant recognizes that WindowPoSt's DoPartition function is the next piece to understand, indicating a systematic, breadth-first exploration strategy.

Second, the assistant articulates the three-step implementation plan: update configuration, create the gRPC client, and wire the delegation. This is the skeleton that the formal plan will flesh out. The assistant expresses confidence ("I've crafted a plan, which I think is pretty solid"), which is notable for an AI system that typically hedges its statements. This confidence is earned — the plan is indeed well-structured and grounded in the codebase exploration.

Third, the assistant focuses on the delegation of scheduling and backpressure, which it correctly identifies as "the biggest architectural challenge." The reasoning here is precise: TypeDetails() must be modified to bypass local limits, and CanAccept() must be modified to query the daemon. The assistant uses the phrase "effectively forcing the harmonytask orchestrator to not limit tasks," which captures the essence of the design.

What is notably absent from the reasoning is any discussion of failure modes. The plan does not address what happens if the daemon becomes unreachable during task execution, how to handle partial failures (e.g., some proofs succeed while others fail), or how to recover from a daemon crash. These are significant omissions that would need to be addressed before production deployment, but they are appropriate to defer from a planning document focused on initial integration.

Assumptions Embedded in the Plan

Every architectural plan rests on assumptions, and message 3366 is no exception. Identifying these assumptions is crucial for understanding the plan's strengths and limitations.

Assumption 1: The daemon's queue depth is a sufficient proxy for resource pressure. The plan uses MaxPending as the sole backpressure signal, assuming that each queued proof consumes a predictable amount of GPU memory and that the queue depth alone determines whether new work can be accepted. This ignores the possibility that different proof types have different resource footprints, or that GPU memory fragmentation could cause failures even when the queue is shallow.

Assumption 2: Curio's scheduler will continue to propose tasks even when resources are zeroed. The plan sets GPU and RAM costs to zero in TypeDetails(), which tells the scheduler that the task requires no local resources. The assumption is that the scheduler will then propose the task freely, relying entirely on CanAccept() for backpressure. If the scheduler has a minimum resource threshold below which it refuses to schedule tasks (e.g., it might require at least some RAM), this approach could backfire.

Assumption 3: The gRPC connection can be long-lived and auto-reconnecting. The plan assumes that a single persistent connection to the daemon is sufficient and that auto-reconnect logic can handle transient failures. This is reasonable for a local daemon communicating over a Unix socket, but it may not hold if the daemon is on a remote machine with network interruptions.

Assumption 4: The protobuf definitions are stable and complete. The plan references ProofKind values like POREP_SEAL_COMMIT, SNAP_DEALS_UPDATE, and WINDOW_POST_PARTITION without verifying that these exact constants exist in the generated Go code. The assistant had read the protobuf file in message 3363, but the mapping from protobuf enum values to Go constants depends on the protoc generator's conventions.

Assumption 5: The task's Do() method can be modified to call the daemon without restructuring the surrounding control flow. The plan suggests intercepting the computation in Do() and DoPartition(), but it does not address how to handle the existing FFI fallback path. The code snippets show an if config.Cuzk.EnablePoRep guard, implying that the local FFI path remains as a fallback, but the plan does not specify how to maintain both paths without code duplication.

Input Knowledge Required to Understand This Message

To fully grasp message 3366, a reader needs familiarity with several domains:

Curio's Harmony task system: Understanding the harmonytask.TaskType interface, particularly the roles of TypeDetails(), CanAccept(), and Do(). The reader must know that TypeDetails() declares resource costs, CanAccept() gates task admission, and Do() executes the work.

Filecoin proof types: Knowledge of PoRep (Proof of Replication), SnapDeals (sector updates), WindowPoSt (Window Proof of Spacetime), and WinningPoSt. Each proof type has different inputs and serialization formats.

gRPC and protobuf conventions: Understanding how protoc generates Go client stubs, how //go:generate directives work, and how gRPC connections are established and maintained.

Go configuration patterns: Familiarity with how Curio loads configuration from TOML files and how nested structs are used to organize settings.

The cuzk project's history: Knowledge that cuzk is an independent proving daemon with its own GPU scheduler, that it uses a split API (Submit + Await) for latency hiding, and that it was designed to address the memory and throughput bottlenecks identified in earlier phases of the project.

Output Knowledge Created by This Message

Message 3366 produces several forms of knowledge:

An architectural blueprint: The plan itself is the primary output — a structured, four-part design that can be directly implemented. It specifies what files to create, what functions to modify, and what patterns to follow.

A rationale for design decisions: The message explains why each decision was made, not just what to do. The reasoning about zeroing resource costs, consulting the daemon for backpressure, and preserving the existing scheduler semantics provides a design philosophy that guides implementation.

A vocabulary for the integration: The message introduces terms like "backpressure via CanAccept," "zeroed resource costs," and "queue capacity delegation" that become part of the project's shared language.

A prioritization of concerns: By structuring the plan into four pillars, the message implicitly prioritizes configuration and client infrastructure over computation wiring, reflecting a bottom-up integration strategy where the plumbing is established before the endpoints are connected.

A record of deliberation: The "Agent Reasoning" section captures the assistant's thought process, including what it considered, what it deferred, and what it decided. This metacognitive layer is valuable for future developers who need to understand why the architecture took its particular shape.

Significance and Legacy

Message 3366 stands at the inflection point between investigation and implementation in this opencode session. It is the moment when understanding transforms into action. The user's response in message 3367 — simply "Implement" — confirms that the plan was accepted as the roadmap for the next phase of work.

What makes this message remarkable is not the technical depth of the plan (though it is substantial), but the clarity with which it articulates the architectural philosophy of delegation. The plan does not attempt to merge cuzk into Curio or to make Curio aware of the daemon's internal scheduling. Instead, it creates a clean abstraction boundary: Curio's scheduler treats the daemon as a black box with a queue, and the daemon manages its own resources independently. This separation of concerns is the hallmark of good system design.

The plan also demonstrates a mature understanding of how to integrate with an existing scheduler without breaking its semantics. By modifying TypeDetails() and CanAccept() rather than introducing a new scheduling path, the plan preserves Curio's existing task lifecycle while substituting the resource oracle. This is a textbook example of the Open-Closed Principle: the scheduler is open for extension (via new CanAccept implementations) but closed for modification.

In the broader context of the cuzk project, message 3366 represents the bridge between the proving daemon's internal architecture (which was the focus of earlier segments) and its integration into the production orchestrator. The plan ensures that the months of work optimizing GPU memory bandwidth, implementing split APIs, and designing memory backpressure can be leveraged by Curio operators without requiring changes to the daemon's core.

Conclusion

Message 3366 is a masterclass in integration planning. It takes a complex, multi-layered problem — connecting an independent GPU proving daemon to a task orchestrator with its own scheduling semantics — and decomposes it into four coherent pillars, each with clear rationale, code-level detail, and architectural justification. The plan balances ambition with pragmatism, proposing a design that is both comprehensive and implementable.

The reasoning embedded in the message reveals a systematic investigator who explored the codebase, identified the core architectural tension, and designed a solution that respects the boundaries of both systems. The assumptions, while not exhaustive, are reasonable for a planning document and reflect a deep understanding of the systems involved.

For anyone studying how to design integration architectures, message 3366 offers a valuable case study in delegation, abstraction, and the art of keeping two complex systems loosely coupled while making them work together seamlessly.