The Art of the Small Decision: Why "Extra Context Is Always Useful" Transformed an Agent Diagnostic System

"Also add the fallback data to the successful SSH case too (extra context is always useful)"

This single sentence, embedded in an edit command to a Go source file, is one of those deceptively small messages that reveals the entire philosophy behind a system architecture. At first glance, message [msg 4915] appears trivial — a developer adding a few extra fields to a JSON response. But this message sits at the convergence of a much larger story: the construction of a diagnostic grounding system for an autonomous LLM-driven fleet management agent, the discovery that SSH connectivity is unreliable across a distributed GPU proving cluster, and the architectural decision to make the diagnostic endpoint equally informative regardless of whether its primary data source succeeds or fails.

The Context: Building a Diagnostic Sub-Agent

To understand why this tiny edit matters, we must trace the events that led to it. The conversation leading up to [msg 4915] reveals a crisis of trust. The autonomous fleet agent — an LLM tasked with managing a cluster of GPU instances running cryptographic proving workloads — had made a catastrophic decision: it killed multiple instances that were progressing perfectly fine. The root cause was not malice or incompetence, but speculation. The agent saw instances in states it didn't understand and guessed at their health, guessing wrong.

The user's response in [msg 4894] was sharp and directive: the agent must never speculate about instance health. It must ground itself in facts first. This led to the creation of a diagnostic sub-agent system — a two-layer architecture where a Go backend endpoint (GET /api/agent/diagnose/{vast_id}) SSHes into instances to collect logs, process lists, memory stats, and GPU information, and a Python sub-agent LLM interprets that raw data with domain knowledge about normal startup sequences versus actual failures. The stop_instance tool was gated with an HTTP 428 precondition, forcing the main agent to call diagnose_instance first before it could take destructive action.

The Discovery: SSH Is Not a Reliable Foundation

The assistant deployed the diagnostic endpoint and immediately discovered a critical problem: SSH connectivity to the vast.ai GPU instances was unreliable. Some instances were reachable, others returned "Permission denied (publickey)" despite the SSH key being correctly configured on the vast.ai account. The user confirmed in [msg 4907] that the key was set up — some instances simply had broken SSH configurations.

This presented a fundamental challenge. If the diagnostic endpoint could only return useful data when SSH worked, the sub-agent system would be useless for a significant fraction of the fleet. An agent that cannot diagnose instances it cannot SSH into would be forced back into speculation — exactly the behavior the system was designed to prevent.

The assistant's response in [msg 4912] was to design a fallback strategy with three tiers: check the cuzk status API via the portavailc tunnel (for instances that had registered), use the vast.ai API data already available (memory usage, GPU utilization, status), and report what the manager database knows (state, age, last status update). This ensured that even when SSH failed, the diagnostic endpoint would return something — a structured set of signals the sub-agent LLM could interpret.

The Subject Message: Symmetric Data Enrichment

This brings us to [msg 4915]. The assistant had just applied an edit to enhance the SSH-failed fallback path (in [msg 4914]), adding vast API data and database state to the response returned when SSH fails. Then came the subject message:

"Also add the fallback data to the successful SSH case too (extra context is always useful)"

The reasoning is crystalline in its simplicity. If the SSH-failed path benefits from extra context — the vast API data, the database state, the billing rate, the uptime, the GPU model — then the SSH-success path benefits from it too. The assistant recognized that the diagnostic endpoint should not have two tiers of quality: a "rich" response when SSH works and a "poor" response when it doesn't. Both paths should return the same enriched data structure, with SSH providing additional depth on top.

This is a design principle that extends far beyond this single endpoint. When building systems that feed data to LLMs for decision-making, the quality and completeness of the input directly determines the quality of the output. The assistant understood that the sub-agent LLM would make better decisions if it had all available context — not just the logs and processes from SSH, but also the billing metadata, the historical state transitions, the GPU model information. These are signals the LLM can cross-reference: "The logs show a crash loop, but the instance is only 30 minutes old and this GPU model is known to have driver issues — this is likely a transient problem, not a hardware failure."

Architectural Significance

The decision in [msg 4915] reflects a deeper architectural philosophy: symmetric information surfaces. In distributed systems where data sources have variable reliability, the natural tendency is to optimize for the common case and let the fallback be minimal. The assistant rejected this. Instead, it built a diagnostic endpoint where the data structure is consistent regardless of whether SSH succeeded or failed. The SSH path adds more fields (logs, process lists, detailed memory breakdowns), but the base structure — vast API data, database state, instance metadata — is always present.

This symmetry has profound implications for the sub-agent LLM. It means the diagnostic tool's output has a predictable schema regardless of connectivity. The sub-agent can always ask the same questions: "What is the instance's state? How long has it been running? What GPU model is it? What is its billing rate?" The answers may be more or less detailed depending on SSH availability, but the questions are always answerable. This consistency reduces the cognitive load on the LLM and prevents it from having to branch its reasoning based on data availability — a common source of errors in LLM-based decision systems.

Input and Output Knowledge

The input knowledge required to make this decision was substantial. The assistant needed to understand:

Assumptions and Potential Mistakes

The assistant made a reasonable assumption: that the additional context (vast API data, database state) would not conflict with or confuse the SSH-collected data. In practice, this is likely true — the vast API reports "actual_status" while SSH reports running processes, and these are complementary signals. However, there is a subtle risk: if the vast API data is stale or inaccurate (e.g., reporting an instance as "running" when it has actually crashed), the sub-agent LLM might weight the API data over the SSH logs. The assistant mitigated this by having the diagnostic endpoint collect both sources and letting the sub-agent LLM reconcile them, but the risk of hallucinated reconciliation remains.

Another assumption is that "extra context is always useful." This is generally true for LLM-based reasoning, but there is a counterargument: too much context can dilute signal, introduce noise, or cause the LLM to focus on irrelevant details. The assistant implicitly assumed that the added fields (billing rate, GPU model, uptime) are relevant to diagnostic decisions, which they are — but the assumption should be validated empirically.

Conclusion

Message [msg 4915] is a masterclass in the importance of small architectural decisions. In a system designed to prevent an autonomous agent from making destructive decisions based on speculation, the quality of the diagnostic data is the single most important factor. By ensuring that the diagnostic endpoint returns enriched data regardless of SSH connectivity, the assistant built a system where the sub-agent LLM always has the context it needs to make grounded judgments. The phrase "extra context is always useful" is not just a casual remark — it is a design principle that elevates the entire diagnostic architecture from a brittle SSH-dependent tool to a robust, multi-source intelligence-gathering system. In the high-stakes world of autonomous fleet management, where a single bad decision can cost hours of proving time and hundreds of dollars in GPU rental, this kind of principled engineering is what separates a toy from a production system.