The Projected Capacity Fix: Teaching an Autonomous Agent to See the Future

In the sprawling development of an autonomous LLM-driven fleet management agent for cuzk proving infrastructure, one message stands out as a turning point. Message 4503 is deceptively short — a deployment command, a service restart, and a verification curl. But behind this compact sequence lies the resolution of a fundamental cognitive failure in the agent's decision-making architecture. This message represents the moment when the agent was taught to see not just what its fleet is producing, but what it will be producing, transforming it from a reactive over-provisioner into a forward-looking capacity planner.

The Context: An Agent That Couldn't Do the Math

To understand why message 4503 matters, we must understand the crisis that preceded it. The user had directed the assistant to build a fully autonomous fleet management agent ([msg 4486]). The agent's job was to observe Curio SNARK demand on the Filecoin network, scale GPU instances up and down on vast.ai, and alert humans when necessary. It ran on a 5-minute timer, fetched fleet state via a Go API, and used an LLM (qwen3.5-122b) to make scaling decisions.

The agent was working — but it was working wrong. By the time the user asked the assistant to "look at how the agent is doing, if what it's doing makes sense" ([msg 4486]), the agent had launched 8 instances, 7 of which were still in the "loading" state (bootstrapping, downloading proofs, running benchmarks). Only 1 instance was actually producing proofs at a rate of 40 proofs/hour. The agent's target was 500 proofs/hour. It saw a gap of 460 proofs/hour and kept launching more instances, burning through its 5 LLM iterations trying to work around vast.ai's rate limit.

The root cause was stark: the agent's fleet API reported capacity_proofs_hr: 40.1 — the current production rate from running instances. It completely ignored the 6 loading instances that would soon contribute ~300 proofs/hour. The LLM, presented with a 40/500 gap, rationally decided to launch more machines. It had no way to know that 6 more were already on their way. This wasn't a bug in the LLM's reasoning; it was a bug in the information the LLM was given.

The Diagnosis and Design Decision

The assistant's diagnostic message ([msg 4489]) reveals a clear chain of reasoning. The assistant examined the agent's behavior and identified three distinct problems:

  1. Capacity blindness: The agent saw 40/500 and ignored 6 loading instances worth ~300 p/h.
  2. Iteration waste: The agent burned all 5 LLM iterations retrying after rate-limit errors instead of accepting the constraint.
  3. Over-provisioning: 9 vast instances at $3.32/hr when loading capacity already exceeded the target. The assistant's key insight was that the LLM was not making a reasoning error — it was making a perceptual error. The fleet API was hiding critical information. The solution was not to change the LLM's prompt to say "remember to think about loading instances" (a fragile approach that would rely on the LLM's unreliable arithmetic), but to make the projected capacity an explicit, computed field in the API response. This is a design philosophy that recurs throughout the session: never ask the LLM to infer what the system can compute. The assistant decided to add a projected_proofs_hr field to the FleetTotals struct in the Go API (agent_api.go). This field would sum the bench rates of running instances plus an estimated rate for loading instances (using a default or average benchmark rate). The fast-path logic in the Python agent would switch from checking capacity_proofs_hr to checking projected_proofs_hr. The system prompt would be updated to explain projected capacity and instruct the agent to stop launching when projected capacity meets or exceeds the target. And the tool execution loop would be modified to detect HTTP 429 rate-limit responses and break out immediately rather than retrying.

The Implementation: Three Edits, One Deployment

The assistant executed this plan through three targeted edits. First, the Go API was modified to compute projected_proofs_hr alongside capacity_proofs_hr ([msg 4492]). The edit added logic to estimate each loading instance's contribution using a default benchmark rate (e.g., 50 proofs/hour for an RTX 5090), giving the agent a realistic picture of future capacity.

Second, the Python agent's fast-path logic was rewritten to use projected_proofs_hr ([msg 4497]). The fast-path is a critical optimization: when the fleet already meets or exceeds the target, the agent can skip the expensive LLM call entirely and simply report "No action needed." Previously, the fast-path checked capacity_proofs_hr, which was always low when instances were loading, causing the agent to always invoke the LLM. With projected_proofs_hr, the fast-path correctly identifies when enough capacity is coming and avoids unnecessary LLM invocations.

Third, the system prompt was updated to include explicit instructions about projected capacity and rate-limit handling ([msg 4498]). The prompt now told the agent: "If projected capacity (running + loading) already meets or exceeds the target, do NOT launch new instances." It also instructed: "If you receive a rate-limit error (429), accept it and stop — do not retry with different offers." A fourth edit added rate-limit detection in the tool execution loop ([msg 4501]), causing the agent to break out of its iteration loop when a launch returns 429.

The Verification: Message 4503

Message 4503 is the deployment and verification of these changes. The assistant first confirms that both codebases compile cleanly:

python3 -c "import py_compile; py_compile.compile(...)" && echo "PYTHON OK"
PYTHON OK

cd /tmp/czk && go build -o vast-manager-agent ./cmd/vast-manager/ && echo "BUILD OK"
BUILD OK

The "Both clean" comment at the start of message 4503 refers to this dual verification — both the Python agent script and the Go backend compiled without errors. This is significant because the Go build had been producing LSP diagnostics about missing imports (false positives from the LSP not understanding the Go module cache), but the actual compilation succeeded.

The deployment sequence is methodical:

  1. Copy binaries: scp transfers the new vast-manager-agent binary and vast_agent.py script to the management host at 10.1.2.104.
  2. Restart service: The vast-manager systemd service is stopped, binaries are replaced, and the service is restarted.
  3. Verify endpoint: A curl command hits the updated /api/agent/fleet endpoint and prints the critical new field. The verification output is the payoff:
Running: 1, Loading: 7
Capacity: 40 p/h, Projected: 390 p/h
Summary: 1 running, 7 loading (~350 p/h when ready). Capacity: 40 p/h now, ~390 p/h projected. $0.43/hr spend, $6.20/hr headroom.

The numbers tell the story. The fleet has 1 running instance (40 p/h actual) and 7 loading instances. The projected capacity of 390 p/h is computed by adding the running instance's bench rate plus estimated rates for each loading instance. The summary now explicitly states "~350 p/h when ready" — the human-readable form of the same information. With a target of 500 p/h, the agent now sees 390 p/h projected instead of 40 p/h actual. It still has room to grow, but the gap is dramatically smaller, and the agent will not panic-launch more instances.

Assumptions and Their Validity

The assistant made several assumptions in this fix. First, it assumed that loading instances will eventually reach their estimated benchmark rate. This is reasonable — the benchmark rate is measured during the instance's bench_done phase, and instances that reach "running" state typically achieve their benchmarked throughput. However, there's no guarantee: a machine could fail its benchmark, get stuck in loading, or underperform due to thermal throttling or GPU driver issues. The projected capacity is an estimate, not a guarantee.

Second, the assistant assumed that a default benchmark rate (50 p/h for RTX 5090) is a reasonable stand-in for instances that haven't completed benchmarking yet. This is a pragmatic assumption — without actual benchmark data, some estimate is better than zero. The risk is overestimating capacity for slow or faulty machines, but the agent's fast-path will still invoke the LLM when projected capacity is close to the target, allowing the LLM to make a more nuanced judgment.

Third, the assistant assumed that rate-limit errors (429) are always transient and should cause the agent to stop immediately. This is correct for vast.ai's rate limiting, which enforces a 3-launches-per-15-minute window. Retrying within the same window is futile. However, this assumption might not hold if the rate limit is per-endpoint rather than global — the agent might be able to call other endpoints while launch is rate-limited. The current implementation breaks out of the entire iteration loop, which is conservative but could prevent the agent from taking other actions (like stopping instances) during the cooldown period.

Input and Output Knowledge

To understand message 4503, one needs knowledge of several domains. The reader must understand the architecture: a Go backend (vast-manager) serves a REST API, a Python agent script (vast_agent.py) runs on a cron timer and calls the API, and the agent uses an LLM to make scaling decisions on vast.ai. One must understand the concept of "loading" instances — machines that have been provisioned but are still bootstrapping (downloading proofs, running benchmarks) and not yet producing proofs. One must understand the rate-limit mechanism on vast.ai, which restricts how frequently new instances can be launched. And one must understand the fast-path optimization in the agent, which skips the LLM call when the fleet already meets its capacity target.

The message creates new knowledge: the fleet API now exposes projected_proofs_hr as a first-class field, giving both the agent and any human operator a clear picture of incoming capacity. The agent's behavior is now governed by projected rather than actual capacity, preventing the over-provisioning that plagued the earlier version. The rate-limit handling is hardened — the agent will no longer waste LLM iterations on futile retries. And crucially, the verification proves that the fix works: the endpoint returns 390 p/h projected, and the summary text reflects the new understanding.

The Thinking Process

The assistant's reasoning, visible across messages 4489–4503, follows a clear diagnostic pattern. First, observe: the agent launched too many instances. Second, analyze: the root cause is that the LLM sees only current capacity. Third, design: make projected capacity an explicit API field rather than relying on the LLM to infer it. Fourth, implement: modify the Go API, the Python agent, and the system prompt. Fifth, verify: deploy and test, confirming the fix with real data.

The most sophisticated aspect of this reasoning is the assistant's refusal to solve the problem at the wrong level. A naive fix would have been to add a line to the system prompt: "Remember that loading instances will soon contribute capacity." This would have been fragile — it depends on the LLM correctly performing arithmetic on vague estimates, which LLMs are notoriously bad at. Instead, the assistant pushed the computation into the API layer, where it can be done deterministically, and presented the result as a single number. This is a recurring theme in the session: the system is designed to minimize the cognitive burden on the LLM and maximize the information available in structured, pre-computed form.

Conclusion

Message 4503 is a small message with a large impact. It marks the moment when the autonomous fleet management agent graduated from reactive to predictive capacity planning. The fix was not about changing the LLM's reasoning ability — it was about changing what the LLM could see. By adding projected_proofs_hr to the API, the assistant gave the agent the one piece of information it was missing: the future. The agent could finally see that 7 loading instances were not 7 idle machines burning money, but 7 machines about to contribute 350 proofs/hour. The over-provisioning stopped, the rate-limit retries stopped, and the agent began making decisions that aligned with the actual trajectory of the fleet rather than its momentary snapshot.