The Art of the Observation String: Designing the Agent's Window into the Fleet

"Also we should research what to put into the cron prompt, imo should be one line per instance of status string (instance xx state xx since xx: xxxx..)"

At first glance, this message from the user ([msg 4849]) appears to be a simple suggestion about formatting. A request to put "one line per instance" in the cron prompt. But in the context of building a fully autonomous LLM-driven fleet management agent for distributed GPU proving infrastructure, this message represents a profound design insight: the agent can only make good decisions if it sees the right information in the right format.

This short message sits at the intersection of prompt engineering, systems observability, and autonomous agent design. It arrived at a pivotal moment in the conversation—immediately after the assistant had just completed a major implementation wave incorporating state-of-the-art techniques across four domains (prompting, tool definitions, context management, and event-driven triggering). The subagent tasks in [msg 4848] had just finished implementing enum-based launch priorities, a systemd.path event trigger, session state anchors, smart context compaction, and a restructured system prompt with critical rules at the end. With the scaffolding in place, the user now turned their attention to the substance of what the agent actually reads each time it wakes up.

The Motivation: Why the Observation String Matters

The user's suggestion was not merely cosmetic. To understand its importance, we must consider the agent's operating cycle. The agent runs on a hybrid trigger model: a 5-minute systemd timer as a backstop, plus a systemd.path unit that fires instantly on P0/P1 events (worker deaths, human messages, state changes). Each time the agent runs, it receives a system prompt containing its objectives, tool definitions, and conversation history—but critically, it also receives an observation string that describes the current state of the fleet.

This observation string is the agent's primary sensory input. It is how the agent "sees" the world before making decisions. If the observation string is too verbose, the agent wastes tokens and attention on irrelevant details. If it is too sparse, the agent makes decisions based on incomplete information—as happened catastrophically earlier in the conversation when the agent misinterpreted active=False and stopped all running instances despite 59 pending tasks, because the demand signal could not distinguish "no demand" from "all workers dead with tasks queued."

The user's insight was that the observation string needed to be compact, structured, and information-dense. One line per instance. Each line containing the instance identifier, its current state, how long it has been in that state, and a status string. This format gives the agent a quick, scannable view of the entire fleet's health and activity.

The Design Decision: What Makes a Good Observation Format?

The user's proposed format—instance xx state xx since xx: xxxx..—embodies several design principles:

Per-instance granularity: The agent needs to reason about individual machines, not just aggregate statistics. A machine stuck in "loading" for 3 hours is a different problem from one that just started loading.
Temporal awareness: Including "since xx" gives the agent a sense of duration. A machine that has been "running" for 2 hours is healthy; one that has been "running" for 10 minutes might still be warming up. This temporal dimension was critical after the earlier incident where the agent killed instances that were still in their startup phase.
Compactness: The user specifically wanted one line per instance. This forces conciseness—no verbose descriptions, no narrative. Just the essential signal.
Status string richness: The "xxxx.." placeholder suggests a status string that captures the machine's current activity or health. This could include proofs-per-hour, GPU utilization, error counts, or other operational metrics. The user's framing of this as a research question—"Also we should research what to put into the cron prompt"—is telling. They recognized that the observation format was not an arbitrary choice but a design parameter that deserved systematic investigation. What information does the agent actually need? What can it derive from tool calls? What should be pre-computed and served in the observation string?

Assumptions Embedded in the Message

The user's message makes several implicit assumptions worth examining:

First, it assumes that the agent's observation string is the right place to invest optimization effort. This is a bet on the importance of the agent's perceptual interface over other aspects of the system. Given the earlier catastrophic failure caused by a poor demand signal, this assumption seems well-founded.

Second, it assumes that a compact, per-instance format is superior to alternative representations (e.g., aggregate statistics, narrative summaries, or raw JSON dumps). The user is implicitly rejecting both overly verbose formats (which waste tokens) and overly abstract formats (which hide important detail).

Third, it assumes that the agent can effectively process this structured, line-oriented format. This is consistent with research showing that smaller models (70B-122B parameters, like the Qwen 3.5-122b model used here) benefit from structured, repetitive formats that reduce cognitive load and make patterns easier to detect.

Fourth, it assumes that the status string should be pre-computed by the backend rather than derived by the agent from tool calls. This is a division-of-labor decision: the Go backend collects raw data from vast.ai and Curio APIs, computes a concise status string, and presents it to the agent. The agent then reasons about this pre-digested information rather than making its own API calls to gather raw data.

The Broader Context: A Pattern of Iterative Hardening

This message belongs to a broader pattern in the conversation of iterative hardening through attention to detail. The agent had already been through several rounds of improvement:

Initial design: Basic agent with launch/stop tools, running on a 5-minute timer.
First crisis: Agent killed all instances despite pending tasks → added demand_queued and workers_dead flags.
Second crisis: Rate limiter blocked emergency launches → added launch_priority enum.
Third crisis: Context overflow and session reset bugs → added compaction, debouncing, and session state anchors.
Current focus: Improving the quality of the agent's perceptual input. The user's message fits into this trajectory as the next logical step: having fixed the mechanics of the agent loop (triggering, context management, tool definitions), it was time to fix the content of what the agent sees. The observation string is the agent's window into the fleet, and the user wanted to make sure that window was clear, not frosted.

What Knowledge Was Required to Understand This Message?

To fully grasp the significance of this message, one needs:

Knowledge of the agent architecture: The agent runs on a cron loop (every 5 minutes) plus event-driven triggers. Each run begins with a system prompt that includes an observation string describing the fleet state.
Knowledge of the earlier failures: The incident where the agent stopped all running instances despite 59 pending tasks, because the demand signal was too coarse. This created the motivation for richer, per-instance observation data.
Knowledge of the SOTA improvements just implemented: The enum-based launch priority, systemd.path triggering, session state anchors, and context compaction. These changes had just been deployed, and the user was now looking at the next frontier.
Knowledge of LLM behavior with structured data: The understanding that smaller models (70B-122B) process structured, repetitive formats better than free-form narrative. This informed the user's preference for one-line-per-instance formatting.
Knowledge of the fleet management domain: Understanding what states an instance can be in (loading, running, exited, error), what "since" means (time in current state), and what status information is operationally relevant.

The Output Knowledge Created

This message generated several forms of knowledge:

Immediate design direction: The observation string should use a per-instance, one-line format with state, duration, and status. This became the specification for the next implementation phase.

Research agenda: The user explicitly framed this as a research question—"Also we should research what to put into the cron prompt"—creating a directive to systematically investigate what information the agent needs.

Implementation priority: By raising this issue immediately after the SOTA improvements were deployed, the user signaled that observation quality was the next bottleneck in agent performance.

Architectural principle: The message established a principle of pre-computed, structured observation data served by the backend, rather than raw data that the agent must interpret through tool calls.

The Thinking Process Revealed

The user's thinking process, visible in this brief message, reveals a sophisticated understanding of autonomous agent design:

Systems thinking: The user sees the agent not as a monolithic intelligence but as a system of components—triggering, context, tools, and perception. Each component needs to be optimized.
Prioritization: The user correctly identified that after fixing the mechanics (triggering, context, tools), the next bottleneck was the quality of the agent's perceptual input. This is textbook bottleneck analysis.
Concrete specification: Rather than saying "improve the observation string," the user provided a specific format: one line per instance, with state, duration, and status. This is a design specification, not just a direction.
Research orientation: The user framed this as a research question, acknowledging that the optimal format is not obvious and deserves systematic investigation. This is a mature engineering approach.
Token economy awareness: The user's preference for compact, one-line-per-instance format shows awareness of the LLM's context window constraints. Every token in the observation string consumes budget that could be used for reasoning or tool results.

Conclusion

The user's message at [msg 4849] is a masterclass in concise, high-leverage design direction. In just 26 words, it established a specification, a research agenda, and an architectural principle for the agent's perceptual interface. It emerged at exactly the right moment—after the mechanical scaffolding was in place but before the team moved on to other concerns. The observation string format proposed here—one line per instance, with state, duration, and status—would go on to shape how the agent perceived and reasoned about the fleet, directly impacting its ability to make sound scaling decisions and avoid the catastrophic failures that had plagued earlier iterations.

In the broader narrative of building a reliable autonomous agent, this message represents the shift from "can the agent act?" to "can the agent see?"—a transition that marks the difference between a brittle prototype and a robust production system.