When the Reset Button Breaks the Agent: A Post-Mortem of Context Overflow and Session State Corruption in an Autonomous LLM Fleet Manager

The Message

So: Agent crossed 31k context (actually on backend was into 38k already), and the compaction (is that even implemented correctly?) didn't run. Then I've clicked the 'Reset' reset-settion button, which did clear the session, However now the model does not run on cron nor does it respond to the 'Trigger run now' button.

This single user message, sent at a critical juncture in the development of an autonomous LLM-driven fleet management agent, reads like a distress signal from the front lines of production operations. In just three sentences, the user reports a cascading failure: context overflow, a broken compaction mechanism, a destructive reset, and complete agent paralysis. The message is terse, urgent, and laced with skepticism ("is that even implemented correctly?"). It is the kind of message that separates a working prototype from a production system — the moment when latent bugs surface under real operational stress.

Context: What Led to This Crisis

To understand why this message was written, one must understand the system it describes. The user and assistant had been building an autonomous agent to manage a fleet of GPU instances on vast.ai, running SNARK proofs for the Curio/CuZK proving system. The agent operates on a 5-minute systemd timer, with an additional event-driven path unit that triggers on state changes. It maintains a rolling conversation log in SQLite, with a 30,000-token context window fed to an LLM (qwen3.5-122b) that makes scaling decisions — launching instances, stopping them, diagnosing failures, and alerting humans.

The message arrives immediately after the assistant deployed two fixes ([msg 4997]): reducing noise from the schedule_next_check tool (making 240–360s calls a no-op since the heartbeat covers that window) and adding a "skipped in prompt" badge in the UI for no-action runs. These were incremental improvements to context management and observability. But they did not address the fundamental fragility of the system.

The user's report reveals that the underlying architecture had three critical failure modes that converged simultaneously: context compaction was unreliable, the session reset was destructive, and the trigger mechanism had no way to recover from a cleared state.

The Three Failures, Unpacked

1. Context Overflow and Compaction Failure

The agent's context had ballooned to 38,000 tokens — nearly 30% over the 30k budget. The user's parenthetical "(is that even implemented correctly?)" is a pointed question that reveals a loss of confidence in a core subsystem. Context compaction was designed to keep the LLM prompt within budget by summarizing older messages, pruning no-op runs, and truncating verbose tool outputs. But it had not fired.

Why? The assistant's subsequent investigation ([msg 5001] onward) revealed that compaction had multiple failure modes. The most insidious was that the compaction threshold was checked against the serialized conversation size, but the token counting used a different estimation method, creating a gap where the system believed it was under budget when it was actually over. Additionally, the compaction logic only ran at the beginning of each agent cycle — if the agent never completed a cycle (because it crashed or was rate-limited), compaction never triggered.

But there was a deeper structural problem: the conversation was growing faster than compaction could keep up because of duplicate runs. The agent was being triggered multiple times in quick succession — once by the 5-minute timer, and again by the event-driven path unit responding to state changes. Each trigger spawned a full agent cycle that appended messages to the conversation. With 20 stale "pending wakes" accumulating in the scheduling system (wake events that were never marked as processed), every cycle would re-process them, generating more tool calls, more results, and more context bloat.

The user's question about compaction being "implemented correctly" was entirely justified. The compaction mechanism existed in theory but had never been tested under the load of a real production conversation with dozens of runs, hundreds of tool calls, and the combinatorial explosion of duplicate event triggers.

2. The Destructive Reset

The user clicked the "Reset Session" button. This action was intended to clear the agent's conversation history — a clean slate when things go wrong. But it had a catastrophic side effect: it broke the agent entirely.

The root cause, as the assistant later discovered, was that the agent's run_id was derived from the conversation history. Specifically, the run counter was computed by counting the number of distinct run_id values in the conversation messages. When the reset cleared all messages, the run counter reset to zero. But the damage went deeper.

The agent's session state — including its objectives, target proofs-per-hour, and fleet snapshot — was stored in the conversation thread as system messages. When the reset cleared those messages, the agent lost all context about what it was supposed to be doing. More critically, the trigger mechanism relied on the existence of a valid session to process incoming events. With an empty conversation and no session state, the trigger path unit would fire, the agent would start, immediately encounter an empty or invalid state, and exit without taking any action — or worse, it would fail silently.

The user's typo — "reset-settion" — is a small but telling detail. It suggests the button was clicked hastily, perhaps in frustration as the context overflow warning appeared. The UI had no confirmation dialog, no warning about consequences, no indication that this action could break the agent loop. It was a nuclear option presented as a simple reset.

3. Complete Agent Paralysis

The most alarming sentence is the final one: "However now the model does not run on cron nor does it respond to the 'Trigger run now' button." The agent was completely dead. Two independent activation mechanisms — the systemd timer and the manual HTTP trigger — both failed to produce a response.

This was not a simple service crash. The systemd units were still active (vast-agent.timer showed "active (waiting)" with a pending trigger). The agent service would start and then... nothing. The assistant's initial diagnostics ([msg 5002]) showed that the conversation had been reduced to 8 messages and 167 tokens — the reset had worked in clearing history, but the agent was stuck in a state where it couldn't proceed.

The paralysis had multiple contributing causes. First, the agent's startup logic checked for a valid session anchor — a system message that contained the current objectives and fleet state. After the reset, this anchor was gone, and the agent had no fallback for re-initializing it. Second, the run_id derivation from conversation history produced run_id=0 (or run_id=1), which conflicted with the agent's internal logic for filtering duplicate messages and tracking state changes. Third, the trigger file — the mechanism by which both the cron timer and the "Trigger run now" button signal the agent to start — was left in a stale state after the reset, causing the path unit to fire but the agent to immediately exit because it detected a "no-op" condition.

The Thinking Process Revealed

The assistant's response to this message ([msg 5001]) shows a structured diagnostic approach. The first action is to create a todo list with four items: inspect current services and logs, inspect compaction logic, fix the trigger/cron execution, and fix compaction behavior. This is immediately followed by three parallel SSH commands to check systemd status, journal logs, and the current conversation state.

The reasoning traces reveal the assistant working through the problem systematically. It checks whether the services are running, whether the trigger path is functional, and what the conversation looks like post-reset. The discovery that the conversation has only 8 messages and 167 tokens confirms the reset worked — but the agent is still not running. This narrows the problem to the trigger mechanism itself, not the conversation content.

The subsequent investigation ([msg 5003]) uncovers the real culprit: "20 stale pending wakes that never get marked processed." Every agent cycle logs "Processing 20 scheduled wakes..." and re-processes them all, generating redundant tool calls and context bloat. The trigger file is being "touched multiple times in a burst," causing repeated event-triggered runs seconds apart. The run_id derivation from conversation history means that after a reset, the agent keeps saying "This is run #1" — harmless for behavior but indicative of a deeper fragility.

Assumptions and Their Consequences

The user made several assumptions that proved incorrect. First, that the compaction mechanism would automatically keep context under budget. Second, that the Reset button was a safe, reversible operation. Third, that the cron timer and manual trigger were independent enough that one would work even if the other failed.

The assistant, in building the system, had made its own assumptions: that compaction thresholds would be sufficient, that session reset would be a rare enough event that deriving run_id from conversation history was acceptable, and that the event-driven trigger path would not produce duplicate runs when combined with the timer.

Both parties learned a hard lesson about the brittleness of state management in autonomous agent systems. The conversation history was being used for too many purposes simultaneously: as the LLM prompt, as the session state store, as the run counter, and as the audit log. When one function was disrupted (clearing history), all other functions broke simultaneously.

The Knowledge Created

This message catalyzed a major debugging and hardening session. The assistant went on to implement:

1. A POST /api/agent/mark-wakes endpoint to clear processed wakes from the pending queue
2. A debounce mechanism in triggerAgent() to suppress burst file modifications (2-second debounce for P0 events, 10-second for P1)
3. Monotonic run_id tracking via persistent session state instead of fragile conversation history derivation
4. Proper wake processing so stale entries don't accumulate and re-trigger every cycle These fixes addressed the root causes rather than the symptoms. The context overflow was caused by duplicate runs from burst triggers and stale wakes. The reset bug was caused by fragile run_id derivation. The agent paralysis was caused by the trigger mechanism having no recovery path from a cleared state.

Conclusion

The user's message at index 4998 is a pivotal moment in the development of this autonomous agent. It is the moment when theoretical robustness meets operational reality. The message reveals that context management, session state, and event triggering are not independent concerns — they are deeply intertwined, and a failure in one cascades into failures in all others. The skepticism in the user's voice ("is that even implemented correctly?") is earned. The compaction mechanism existed but was not tested under realistic load. The reset button existed but was not designed with safety guards. The agent ran but had no recovery path from a cleared state.

Building reliable autonomous agents requires not just implementing features, but stress-testing them under the combinatorial chaos of real production use. This message, and the debugging session it triggered, transformed the agent from a fragile prototype into a hardened system with debounced triggers, persistent session state, and robust wake management. The crisis was the catalyst for that transformation.