From Volatile Signals to Robust Rules: Refactoring an Autonomous Fleet Agent's Decision Core

Introduction

In the rapidly evolving landscape of decentralized GPU proving infrastructure, the difference between a functional autonomous agent and a destructive one often comes down to a single question: what signal does the agent use to make decisions? Message 4464 in this opencode session captures a pivotal moment where an AI assistant summarizes a fundamental redesign of an autonomous fleet management agent's decision-making architecture. The message, written after deploying both Go backend changes and a rewritten Python agent script, documents the transition from a fragile, pending-task-based scaling system to a robust, demand-aware logic that accounts for loading instances, respects rate limits, and tracks per-machine performance.

This article examines message 4464 in depth—not merely as a status update, but as a window into the reasoning, assumptions, and engineering tradeoffs involved in building reliable autonomous infrastructure agents. The message is a summary, but it encodes weeks of iterative debugging, user feedback, and architectural evolution compressed into a few bullet points.

The Message in Its Full Context

To understand why this message was written, one must first understand the journey that led to it. The broader session (Segment 32) began with a production crisis: the budget-integrated pinned memory pool was crashing on constrained machines, and the cuzk daemon supervisor had a fundamental reliability bug where wait -n blocked indefinitely even after the process exited. The user, frustrated by the brittleness of the infrastructure, directed the assistant to build a fully autonomous LLM-driven fleet management agent.

What followed was a rapid prototyping cycle. The assistant built a comprehensive Go API (agent_api.go) with 12 endpoints, a Python autonomous agent (vast_agent.py) that used an LLM to make scaling decisions, and deployed the system to a management host. The initial agent used pending task counts as its primary scaling signal—a decision that proved catastrophically wrong. Pending tasks are highly volatile in a system where instance startup takes 1–2 hours, leading to over-provisioning during transient spikes and under-provisioning during sustained demand.

The user's operational feedback reshaped the agent's logic. Message 4464 is the culmination of that feedback loop: the assistant had just deployed a rewritten agent with simple, robust rules, and this message serves as the official handoff—a structured summary of what changed, why, and what the current state of the fleet looks like.

Quoting the Subject Message

The full text of message 4464 reads as follows:

All deployed and working. Here's the summary of what changed:

### Decision Logic (was → now) - Was: Scale up when pending > 10 (volatile noise) - Now: Scale up when demand.active == true AND fleet capacity < target_proofs_hr (500) - Was: Scale down when idle 30min - Now: Scale down when no completions for 1h+ (demand.active == false) - Was: Ignore loading instances - Now: Fleet summary explicitly says "N loading (will add capacity in 1-2h)"

### New Endpoints - GET /api/agent/perf — per-machine completion/error counts from Curio DB (1h + 24h windows) - GET /api/demand now includes active flag, throughput_15m, improved summary

### Agent Config Changes - target_proofs_hr: 500 (replaces scale_up_threshold_pending) - idle_shutdown_minutes: 60 (replaces scale_down_idle_minutes: 30) - min_instances: 0 (was 2 — don't force instances when there's no demand)

### Performance Tracking - Agent writes /var/lib/vast-manager/fleet-performance.md on every run - Includes: active instances, machine track record from Curio DB, recent agent actions, fleet summary - This file is injected into the LLM prompt so it prefers proven machines

### Rate Limit Fix - Only counts successful launches, not rejected attempts

### Current State - 1 running instance (RTX 4090, 40 p/h) - 2 loading (RTX 5090 instances, from agent launches) - Agent timer running every 5 minutes - Next run will see the loading instances and likely hold off on more launches until they're ready

Deep Dive: The Decision Logic Transformation

The most significant change documented in this message is the shift from pending-task-based scaling to demand-aware scaling. This is not merely a parameter tweak—it represents a fundamentally different philosophy of how an autonomous agent should reason about its environment.

The old approach (scale up when pending > 10) treated the Curio task queue as a direct proxy for demand. The assumption was that pending tasks represent unmet work that requires more instances. In practice, this assumption failed because pending counts are noisy: a single large task decomposition can create hundreds of pending sub-tasks in seconds, while a slow worker can accumulate pending tasks without any actual demand increase. The agent would launch instances into a spike that had already passed by the time the instances came online 1–2 hours later.

The new approach uses a composite signal: demand.active (derived from recent completion throughput) combined with a capacity comparison against a fixed target (target_proofs_hr: 500). This is more robust because:

Throughput is a lagging indicator that smooths out transient spikes
The capacity comparison prevents over-provisioning when existing instances are already sufficient
The fixed target provides a clear, operator-controlled ceiling The assistant's reasoning, visible in the preceding messages ([msg 4458]), shows careful consideration of this tradeoff: "The problem is that it doesn't account for loading instances that will add capacity once ready. But this is actually fine—the user said 'no complex auto-targets'." This reveals a deliberate design constraint: the assistant chose simplicity over precision, trusting the LLM's reasoning ability combined with the explicit loading-instance annotation in the fleet summary.

Assumptions Embedded in the New Design

Message 4464 encodes several assumptions that deserve scrutiny. First, the assumption that demand.active (derived from recent completion throughput) is a reliable proxy for actual demand. This works well when the fleet has at least one running instance to generate completion data, but it creates a cold-start problem: if all instances are down and tasks are queued, throughput drops to zero, and demand.active becomes false—exactly the wrong signal. This failure mode would later manifest catastrophically in Chunk 3, where the agent interpreted active=False as "no demand" and stopped all running instances despite 59 pending tasks, leading to the addition of demand_queued and workers_dead emergency flags.

Second, the assumption that the LLM (qwen3.5-122b) would correctly interpret the fleet summary and not over-provision despite the capacity gap. The assistant's reasoning in [msg 4458] reveals this tension: "The agent will see this and may try to launch more instances because capacity (40) is well below target (500). The problem is that it doesn't account for loading instances that will add capacity once ready." The assistant chose to trust the LLM's reasoning ability rather than hard-coding a capacity projection, a decision that reflected the user's preference for simplicity but introduced reliance on model quality.

Third, the assumption that a 60-minute idle shutdown threshold is sufficient to distinguish between genuine demand cessation and temporary lulls. In a system where proofs can take 10–15 minutes each and completion patterns are bursty, an hour of inactivity could represent either a true demand gap or a statistical fluctuation. The choice of 60 minutes (double the previous 30) was a conservative hedge, but it still assumed that the demand signal would be decisive within that window.

The Rate Limit Fix: A Case Study in Self-Reinforcing Bugs

One of the most instructive details in message 4464 is the rate limit fix: "Only counts successful launches, not rejected attempts." This sounds trivial in retrospect, but the bug it fixed was a classic example of a self-reinforcing failure mode. The rate limiter was counting all launch actions—including rejected ones—in its 15-minute window. Each rejection incremented the counter, making future rejections more likely, creating a positive feedback loop that could permanently block the agent from launching any instances.

The assistant's debugging process in [msg 4452] shows the moment of insight: "The rate limit check is counting ALL launch actions (including rejected ones) in the last 15 minutes. We have 3 successful launches and 6 rejected ones. The rate limit should only count successful launches, or the rejected ones accumulate and permanently block." This is a subtle bug that would be invisible in unit tests—it only manifests under real operational conditions where rejections happen for legitimate reasons (e.g., concurrent launches from the same agent).

The fix required changing a single SQL query in agent_api.go from counting all actions to counting only actions with results starting with "ok:". The assistant deployed this fix in [msg 4453]–[msg 4455], and the subsequent agent run in [msg 4456] confirmed the rate limit was still active (due to legitimate successful launches within the window), but no longer self-reinforcing.

The Performance Tracking Innovation

The fleet-performance.md mechanism deserves special attention as an example of how the assistant solved the cold-start problem for instance selection. When the agent needs to launch a new instance, it must choose among available GPU offers on vast.ai. Without historical data, this choice is essentially random—any machine could be fast or broken. The performance tracking file solves this by persisting per-machine completion and error counts from the Curio database across agent runs.

The key insight here is that the performance data is not used algorithmically (e.g., sorting offers by score) but is injected directly into the LLM prompt as raw text. This is a deliberate architectural choice: instead of building a recommendation engine, the assistant lets the LLM reason about the data in context. This approach is more flexible (the LLM can weigh multiple factors) but also more brittle (it depends on the LLM correctly interpreting the data). The assistant's reasoning in [msg 4450] shows awareness of this tradeoff: "This file is injected into the LLM prompt so it prefers proven machines."

What This Message Reveals About the Agent Architecture

Message 4464 serves as a design document for the agent's decision core. The architecture that emerges is a hybrid: simple, deterministic rules for the fast-path decisions (scale up/down based on demand and capacity), augmented by LLM reasoning for the nuanced choices (which machine to launch, whether loading instances are sufficient). This hybrid approach reflects a pragmatic recognition that fully autonomous agents need guardrails—the deterministic rules prevent catastrophic over-provisioning, while the LLM provides the flexibility to handle edge cases.

The message also reveals the agent's operational rhythm: a 5-minute timer cycle where the agent observes state, makes a decision, and potentially acts. The "Current State" section shows the agent's world model at the moment of writing: 1 running instance, 2 loading, with the expectation that the next cycle will recognize the loading instances and hold off on further launches. This is a remarkably grounded summary—it doesn't claim victory or perfection, but simply states what is and what is expected to happen next.

Conclusion

Message 4464 is far more than a deployment summary. It is a snapshot of an autonomous agent at a critical inflection point, transitioning from a fragile, signal-ignorant design to a robust, demand-aware architecture. The decisions encoded in this message—the shift from pending tasks to throughput, the explicit annotation of loading instances, the rate limit fix, the performance tracking mechanism—represent the hard-won lessons from real operational experience. The assumptions embedded in the design would later prove incomplete (the active=False failure in Chunk 3), but the foundation laid in this message was sound enough to support those subsequent refinements.

For anyone building autonomous infrastructure agents, this message offers a masterclass in the importance of signal selection, the dangers of self-reinforcing bugs, and the value of simple, interpretable decision rules augmented by LLM reasoning. The agent described here is not perfect, but it is grounded—it makes decisions based on real data, accounts for its own past actions, and can be understood and adjusted by a human operator. That, ultimately, is the hallmark of a well-designed autonomous system.