The Grep That Saved the Fleet: How a Simple Search Uncovered an Autonomous Agent's Fatal Blind Spot

The Message

[assistant] Now let me find the system prompt and the fast-path logic:
[grep] def build_system_prompt|def run_agent|Situation needs|fast.path|No action needed|capacity >= target
Found 7 matches
/tmp/czk/cmd/vast-manager/agent/vast_agent.py:
  Line 543: If everything looks fine, say "No action needed" and briefly explain.
  Line 548: def build_system_prompt(agent_config: dict, demand: dict, fleet: dict,
  Line 606: def run_agent() -> None:
  Line 607:     """Main agent logic: observe, decide fast-path or LLM, act."""
  Line 668:     if active and capacity >= target and running > 0:
  Line 671:             "Active demand, fleet at capacity (%.0f/%.0f p/h). No action neede...

At first glance, this appears to be a mundane moment in a coding session: an AI assistant running a grep command to locate specific functions in a Python file. But this message sits at the inflection point of a critical debugging arc — the moment when an autonomous fleet management agent's most dangerous flaw was identified and the path to its correction was charted. The grep results visible in this message are not merely a list of line numbers; they are the skeleton key to understanding why an otherwise intelligent LLM-driven agent was systematically over-provisioning expensive GPU compute instances, burning money while remaining oblivious to its own future capacity.

Context: The Autonomous Agent That Couldn't Count

To understand the gravity of this message, one must understand what preceded it. The assistant had built a fully autonomous LLM-driven fleet management agent for a distributed GPU proving infrastructure called "cuzk." The agent's job was straightforward: monitor Curio SNARK demand on the Filecoin network, and scale a fleet of rented GPU instances (from vast.ai) up or down to match that demand, all while respecting budget constraints and minimizing human intervention.

The agent had been running for some time, and the user asked the assistant to "look at how the agent is doing" ([msg 4486]). What the assistant found was alarming. The agent had launched eight instances, but only one was actually running and producing proofs. Six were still in a "loading" state — bootstrapping, syncing, and initializing — a process that takes 1–2 hours. Yet the agent, seeing only 40 proofs/hour of capacity against a target of 500, kept launching more instances. It was like a warehouse manager who, seeing empty shelves, keeps ordering more pallets without noticing that six trucks are already on their way.

The assistant's analysis in [msg 4489] was brutally clear: "The core issue is that the agent doesn't account for loading instances as future capacity — it sees the 40 p/h gap and keeps launching more, when those 6 loading instances will eventually provide ~300 p/h." The agent was also burning through all five of its LLM iterations trying to work around a rate limit instead of recognizing when to stop. The fleet was over-provisioned at $3.32/hour with nine instances when perhaps three or four would have sufficed.

Why This Message Was Written

Message 4494 was written because the assistant had already diagnosed the problem and formulated a two-part solution. The first part — adding a projected_proofs_hr field to the Go backend's fleet API response — had already been implemented in [msg 4492]. The second part required modifying the Python agent itself: updating its fast-path logic and system prompt to use this new projected capacity metric.

But before making any edits, the assistant needed to understand the existing code's structure. The grep command in this message is a reconnaissance mission — a surgical probe into the Python agent to locate exactly the code that needed to change. The assistant needed to find:

  1. The fast-path logic (line 668): The if active and capacity >= target and running > 0 check that decided whether the agent could skip the expensive LLM call and declare "no action needed." This was the code that was failing because it used capacity (only running instances) instead of projected capacity.
  2. The system prompt builder (line 548): The function that constructed the LLM's context, which needed to be updated to include projected capacity information so the LLM could make informed decisions.
  3. The "No action needed" message (line 543, 671): The template strings that the agent used when it decided the fleet was adequately provisioned.
  4. The run_agent function (line 606): The main entry point where the fast-path decision was made. The assistant's reasoning was methodical: fix the data source first (the Go API), then fix the consumer (the Python agent). But before fixing the consumer, you must understand its current behavior. The grep was the bridge between diagnosis and treatment.

Input Knowledge Required

Understanding this message requires several layers of context. First, one must understand the architecture: a Go backend (vast-manager) that serves a REST API and manages a SQLite database, paired with a Python agent (vast_agent.py) that runs on a cron timer, fetches fleet state from the API, and uses an LLM to make scaling decisions. The agent has a "fast-path" optimization: if demand is active and the fleet already meets the target capacity, it can skip the LLM call entirely and log a brief message.

Second, one must understand the bug: the capacity variable in the fast-path check only counted proofs per hour from instances in the running state. Instances in registered or params_done (loading) states contributed zero to capacity, even though they would be producing proofs within 1–2 hours. This meant the agent saw a perpetual capacity deficit and kept launching.

Third, one must understand the fix strategy: introduce projected_proofs_hr — a field that adds estimated capacity from loading instances (using a default benchmark rate) to the actual capacity from running instances. Then update the fast-path and prompt to use this projected figure.

The grep patterns themselves encode this understanding. The assistant searched for capacity >= target because that was the exact comparison that was wrong. It searched for fast.path (with a wildcard) because the fast-path logic was the critical decision point. It searched for No action needed because that was the desired outcome when the fleet was adequately provisioned — a state the agent could never reach under the buggy logic.

The Thinking Process Visible in the Message

Though the message is short, it reveals a sophisticated diagnostic process. The assistant doesn't just grep for a single pattern; it constructs a compound regex that targets five distinct aspects of the code simultaneously:

What Happened Next

The grep results in message 4494 directly informed the edits that followed. In [msg 4495], the assistant read the run_agent function to see the full fast-path logic. In [msg 4497], the assistant declared: "Now I see the issues clearly. Let me fix: 1. Fast-path should use projected_proofs_hr instead of capacity_proofs_hr. 2. System prompt needs to tell the LLM about projected capacity and to not launch when projected >= target. 3. Add rate-limit awareness — if rate-limited, accept and stop."

The edit was then applied successfully ([msg 4497]), followed by a system prompt update ([msg 4498]). The fix was surgical and targeted, thanks to the precise location data obtained from the grep.

Assumptions and Potential Mistakes

The assistant made several assumptions in this message. It assumed that the fast-path logic was the primary culprit in the over-provisioning behavior — that fixing the capacity comparison would be sufficient to prevent the agent from launching unnecessary instances. This was a reasonable assumption, but it overlooked a deeper issue: even with projected capacity in the fast-path, the LLM could still decide to launch instances if the system prompt didn't adequately constrain it. The assistant recognized this and updated the prompt as a follow-up.

Another assumption was that a default benchmark rate for loading instances was a reasonable estimate. Loading instances might have different GPUs with different performance characteristics, and using a single default could either overestimate or underestimate their future capacity. The assistant mitigated this by using the average benchmark rate from the fleet performance file when available, falling back to a default only when no data existed.

The assistant also assumed that the rate-limit problem was separate from the capacity projection problem. In reality, they were intertwined: the agent kept launching because it saw a capacity deficit, and it hit the rate limit because it launched too many times. Fixing the capacity projection would naturally reduce the number of launch attempts, making the rate-limit issue less acute. But the assistant addressed both independently, which was a prudent defense-in-depth approach.

Output Knowledge Created

This message created knowledge in two forms. First, it produced a concrete map of the code that needed to change — seven line numbers across two functions and two message templates, each tagged with its role in the agent's decision-making pipeline. This map directly enabled the subsequent edits.

Second, and more importantly, this message crystallized a design principle: an autonomous agent's perception of its own state must include not just the present but the near future. The agent was not wrong to see 40 proofs/hour of capacity; it was wrong to act as if that number would remain static. By introducing projected capacity, the assistant taught the agent to account for the pipeline of instances that were bootstrapping but not yet productive. This is a general insight that applies to any autonomous system managing resources with non-trivial startup times — whether GPU instances, cloud servers, or even manufacturing equipment.

The message also implicitly validated the architecture of having a fast-path bypass. The fast-path existed to save LLM tokens and latency when the fleet was stable. But a buggy fast-path that never triggered was worse than no fast-path at all — it meant the agent always fell through to the LLM, which then made poor decisions based on incomplete data. Fixing the fast-path restored its value as an optimization while making it correct.

Conclusion

Message 4494 is a study in how the most impactful moments in a debugging session are often the quietest. A single grep command, six patterns, seven matches — and yet this is the moment when an over-provisioning autonomous agent was diagnosed and the path to its correction was laid bare. The message reveals a developer (or in this case, an AI assistant) who understands that before you can fix a bug, you must first find it; and before you can find it, you must know exactly what you're looking for. The compound regex in this grep is not just a search — it's a hypothesis expressed as a pattern, a theory of the bug encoded in regular expressions. And when the matches came back, they confirmed the theory and enabled the fix.