The Projected Capacity Fix: Teaching an LLM Agent to Do Arithmetic

Introduction

In the development of autonomous AI systems, few challenges are as deceptive as the "obvious math problem." When a human operator looks at a fleet of 1 running instance producing 40 proofs per hour alongside 6 instances still bootstrapping, they intuitively understand that those 6 loading machines represent ~300 proofs per hour of incoming capacity. But an LLM-based agent, consuming a text summary of fleet state, has no such intuition—it sees a gap between 40 and 500 and concludes it must launch more. This is precisely the failure mode diagnosed and addressed in message [msg 4489] of a coding session building an autonomous fleet management agent for a distributed proving infrastructure.

The message is a turning point in the agent's development. It represents the moment the assistant stepped back from the operational trenches—fixing crashes, deploying UI panels, building diagnostic systems—and performed a critical meta-analysis of the agent's decision-making logic. What it found was a fundamental cognitive blind spot: the LLM could read the words "6 loading (will add capacity in 1-2h)" but could not incorporate that information into its scaling arithmetic. The fix was not a prompt tweak but a structural change to the data the agent consumes, adding an explicit projected_capacity_proofs_hr field that does the math for the model.

This article examines message [msg 4489] in depth: the reasoning that led to it, the assumptions it challenged, the decisions it encoded, and the thinking process visible in its structured analysis. It is a case study in the subtle art of building reliable LLM agents—where the hardest problems are often the ones that look easiest.

Context: The Autonomous Agent's Growing Pains

To understand message [msg 4489], we must first understand what came before. The session's broader arc (Segment 32 of the conversation) describes the construction of an autonomous LLM-driven fleet management agent for a GPU proving cluster running CuZK and Curio software. The agent's job is to monitor SNARK demand, launch GPU instances on vast.ai to meet that demand, and shut down idle instances to control costs.

In the messages immediately preceding [msg 4489], the assistant had been iterating rapidly on the agent's capabilities. The user had requested UI panels for demand monitoring and agent activity tracking ([msg 4465]), which the assistant implemented by adding "Curio Demand" and "Agent Activity" panels to the vast-manager web UI ([msg 4475]-[msg 4485]). Then the user asked a pivotal question ([msg 4486]):

"Look at how the agent is doing, if what it's doing makes sense, maybe review prompt optimality"

This was not a bug report or a feature request—it was an invitation to reflect. The user suspected something was off and asked the assistant to audit the agent's behavior. The assistant responded by gathering data: journalctl logs of recent agent runs ([msg 4487]) and a comprehensive snapshot of the fleet state ([msg 4488]).

The data told a clear story. The agent had launched 8 instances total. Of those, only 1 was actually running and producing proofs (a single RTX 4090 doing 40 proofs/hour). 6 were still loading—bootstrapping, registered in the database, but not yet contributing. The fleet summary explicitly stated: "6 loading (will add capacity in 1-2h)." Yet the agent, seeing a target of 500 proofs/hour and current capacity of only 40, kept launching more machines. It was over-provisioning by a factor of 2x, burning budget on instances that wouldn't contribute for hours, while already having enough capacity in the pipeline.

The Subject Message: Diagnosis and Decision

Message [msg 4489] is the assistant's response to this audit. It contains two distinct sections: an internal reasoning trace (labeled "Agent Reasoning") and the execution of the fix. The reasoning section is the heart of the message—it is where the assistant performs a structured analysis of what went wrong and why.

The Reasoning Trace

The assistant begins by enumerating the facts:

What happened: 1. Agent launched 8 instances total (3 early with old prompt, 5 with new prompt) 2. 9 vast instances exist now (1 running, 8 loading) 3. Only 1 is actually running and producing proofs (40 p/h) 4. 6 are loading/registered in the DB, still bootstrapping 5. Total DPH: $3.32/hr (7 in DB), but Vast shows 9 total 6. Rate limit is working — blocks after 3 successful launches per 15 min window

This enumeration is itself an act of reasoning. The assistant is not simply reporting what the tools returned—it is synthesizing across multiple data sources (journalctl logs, fleet API responses, SQLite queries, vast.ai CLI output) into a coherent picture. The discrepancy between "7 in DB" and "9 total" is a subtle clue: some instances exist on vast.ai but haven't been registered in the local database yet, suggesting the agent is launching faster than the onboarding pipeline can process.

Then comes the critical insight:

The core issue is that the agent doesn't account for loading instances as future capacity — it sees the 40 p/h gap and keeps launching more, when those 6 loading instances will eventually provide ~300 p/h. It also burns through all its LLM iterations trying to work around the rate limit instead of recognizing when to stop, and it ignores the 1-2 hour startup window mentioned in the prompt.

This is a diagnosis of a cognitive failure, not a software bug. The agent has the information—the fleet summary explicitly says "6 loading (will add capacity in 1-2h)"—but the LLM cannot integrate that information into its decision calculus. The assistant identifies three distinct failure modes:

  1. Capacity blindness: The LLM sees current capacity (40 p/h) and target (500 p/h) and computes a gap of 460 p/h, ignoring the ~300 p/h that will arrive when loading instances finish.
  2. Iteration waste: The agent exhausts its 5 LLM iterations on futile retries after hitting the rate limit, rather than recognizing the limit and stopping.
  3. Temporal blindness: The agent ignores the stated 1-2 hour startup window for new instances, treating them as if they will never arrive.

The Root Cause Analysis

The assistant then performs a deeper causal analysis:

The real problem is that the LLM sees the fleet capacity as just 40 p/h and doesn't mentally add the projected capacity from those 6 loading instances.

This is a profound observation about the nature of LLM-based agents. The model can read the words "6 loading (will add capacity in 1-2h)" but it cannot compute "6 instances × ~50 p/h each = ~300 p/h." The arithmetic is trivial for a human but opaque to a language model consuming a text summary. The assistant correctly identifies that the fix must be structural, not prompt-based: the data itself must be pre-computed.

The Decision

Based on this analysis, the assistant makes a clear decision:

I need to make this calculation explicit by adding a projected_capacity_proofs_hr field to the fleet response that combines running instance rates with estimated rates for loading instances, using average benchmarks or defaults.

This is followed by a three-part action plan:

  1. Add projected_capacity_proofs_hr to the fleet API response
  2. Update the agent's prompt to clearly state loading instances and their projected capacity
  3. Add a rule preventing launches when projected + loading capacity already meets the target
  4. Add a directive to stop immediately when rate-limited instead of wasting iterations The assistant then executes the fix, editing agent_api.go to add the new field.

Assumptions and Their Implications

Message [msg 4489] rests on several key assumptions, some explicit and some implicit.

Assumption 1: Loading instances will eventually become productive. The assistant assumes that each loading instance will eventually contribute its benchmark rate to total capacity. This is reasonable but not guaranteed—instances can fail during bootstrapping, have hardware issues, or be terminated by vast.ai. The assistant implicitly assumes a ~50 p/h per loading instance (based on the RTX 5090 benchmark rate), which may be optimistic if the instances are of different GPU types.

Assumption 2: The LLM's cognitive limitation is the binding constraint. The assistant assumes that the agent's over-provisioning is caused by the LLM's inability to do arithmetic, not by other factors like prompt ambiguity, tool definition issues, or bugs in the decision loop. This is a well-supported assumption given the evidence, but it is worth noting that the assistant does not consider alternative hypotheses (e.g., that the agent is correctly interpreting the data but has a different objective, like maximizing instance diversity).

Assumption 3: Pre-computing the projection is sufficient. The assistant assumes that adding projected_capacity_proofs_hr to the API response will fix the behavior. This assumes the LLM can correctly use a pre-computed number when it cannot compute the number itself. This is a reasonable assumption—models are generally better at reading and acting on explicit numbers than performing multi-step reasoning—but it is not tested in this message.

Assumption 4: The rate-limit behavior is a separate problem. The assistant identifies the iteration-waste issue as distinct from the capacity-blindness issue, and addresses it with a separate prompt directive. This is correct: the two failure modes have different root causes and require different fixes.

Mistakes and Incorrect Assumptions

While the analysis in [msg 4489] is largely sound, there are potential blind spots worth examining.

Potential mistake: Over-attribution to the LLM's arithmetic ability. The assistant assumes the LLM cannot do the arithmetic, but it is possible that the LLM could do it but was not prompted to do it. The prompt might not have explicitly instructed the model to compute projected capacity. If this is the case, the structural fix (pre-computing the projection) is still the right solution—it is more robust than relying on prompt engineering—but the diagnosis of "can't do arithmetic" might be slightly off. The real issue may be "doesn't know it should do arithmetic."

Potential mistake: Ignoring the 8th instance. The assistant notes that "9 vast instances exist now" but only 7 are in the DB, and the fleet summary says "1 running, 6 loading." This leaves 2 instances unaccounted for. The assistant does not investigate this discrepancy, instead focusing on the over-provisioning problem. These orphan instances could represent failed launches, instances that were terminated but not cleaned up, or a race condition in the registration pipeline. Ignoring them might allow a subtle bug to persist.

Potential mistake: No validation of the fix. The assistant edits agent_api.go and deploys it, but does not verify that the new projected_capacity_proofs_hr field actually changes the agent's behavior. The message ends with the edit being applied; there is no follow-up test showing the agent making a better decision. This is a gap in the scientific method—the hypothesis is implemented but not validated.

Input Knowledge Required

To fully understand message [msg 4489], the reader needs knowledge of several domains:

Output Knowledge Created

Message [msg 4489] produces several forms of knowledge:

  1. A diagnosis of LLM agent cognitive failure: The insight that LLMs cannot implicitly compute projected capacity from textual descriptions, even when the information is explicitly stated. This is a generalizable finding for anyone building LLM-based agents that make resource allocation decisions.
  2. A design pattern for fixing it: The solution of pre-computing derived metrics in the API response, rather than relying on the LLM to perform multi-step reasoning. This is the "don't make the model do math" principle, which applies broadly to agent design.
  3. A taxonomy of agent failure modes: Capacity blindness, iteration waste, and temporal blindness are three distinct failure patterns, each requiring a different intervention.
  4. A concrete code change: The projected_capacity_proofs_hr field added to agent_api.go is a specific, deployable artifact that changes the agent's behavior.
  5. A methodology for agent auditing: The message demonstrates a repeatable process: gather operational data, enumerate facts, identify discrepancies, trace root causes to cognitive limitations, and implement structural fixes. This methodology is itself a valuable output.

The Thinking Process

The reasoning visible in [msg 4489] reveals a sophisticated analytical process. Let me trace it step by step.

Step 1: Data gathering. The assistant begins by collecting raw data from multiple sources: journalctl logs show what the agent actually did; the fleet API shows current state; the SQLite database shows launch history; the vast.ai CLI shows the provider's view. This multi-source approach is critical because each source reveals a different facet of the problem.

Step 2: Fact enumeration. The assistant lists six concrete facts. This is a deliberate cognitive technique—by writing down what is known, the assistant creates a stable foundation for analysis. The facts are ordered from most general (8 instances launched) to most specific (rate limit working).

Step 3: Gap identification. The assistant notices discrepancies: 9 instances on vast.ai but only 7 in the DB; 1 running but 6 loading; $3.32/hr spend but only $0.43/hr from running instances. These gaps point to the core problem: money is being spent on instances that aren't producing.

Step 4: Root cause analysis. The assistant identifies the cognitive blind spot: the LLM cannot integrate loading-instance information into its capacity calculation. This is not a bug in the code but a limitation of the model architecture.

Step 5: Solution design. The assistant designs a structural fix: pre-compute projected capacity in the API. This is chosen over prompt engineering because it is more reliable—it doesn't depend on the LLM's reasoning ability.

Step 6: Implementation. The assistant edits agent_api.go to add the new field. The LSP errors shown at the bottom of the message are false positives (missing metadata for imports), not actual compilation errors.

Conclusion

Message [msg 4489] is a masterclass in LLM agent debugging. It demonstrates that the hardest problems in autonomous systems are often not software bugs but cognitive mismatches between what the model can do and what the task requires. The assistant's diagnosis—that the LLM can read "6 loading" but cannot compute "~300 p/h incoming"—is a profound insight that applies far beyond this specific context.

The fix is elegant: instead of trying to teach the model arithmetic through prompt engineering, the assistant changes the data to do the arithmetic upfront. This is a recurring pattern in reliable agent design: meet the model where it is, don't expect it to meet you. The projected_capacity_proofs_hr field is a small change with large consequences—it transforms the agent from a system that launches instances until it hits rate limits into one that can accurately assess whether it already has enough capacity in the pipeline.

The message also demonstrates a rigorous methodology for agent auditing: gather data from multiple sources, enumerate facts, identify discrepancies, trace root causes, design structural fixes. This methodology is reusable across any LLM-based agent system and is perhaps the most valuable output of the message.

In the broader arc of the session, this fix was a turning point. Subsequent messages show the agent making better scaling decisions, and the user's feedback shifts from "fix the over-provisioning" to more nuanced concerns about context management and operational stability. The projected capacity fix didn't just solve a bug—it elevated the agent from a liability to a useful tool.