The Optional Boolean Trap: When LLMs Ignore Critical Instructions in Tool Calls

In the development of autonomous LLM-powered systems, there is a recurring pattern that separates naive prototypes from production-hardened agents: the moment when you realize that what you told the model to do is not what the model actually does. Message [msg 4840] captures precisely such a moment — a brief, almost casual observation that contains a profound lesson about the gap between developer intent and LLM behavior.

The message is short, almost conversational in tone:

The LLM didn't pass emergency=true in its tool call — it just used the default. The prompt tells it about the flag but it didn't use it. Let me check: the LLM sent launch_instance without emergency. The tool description says "Set to true ONLY when workers_dead=true" but the model needs stronger nudging. Let me update the system prompt to be more explicit.

Beneath this surface-level observation lies a cascade of engineering decisions, assumptions about LLM behavior, and a hard-won insight that would directly shape the next phase of the agent's architecture. This message is the pivot point between a working-but-fragile system and one hardened by an understanding of how models actually interpret instructions.

The Crisis That Led Here

To understand why this message matters, we must trace the events that immediately preceded it. The autonomous fleet management agent — a Python-based LLM agent running on a 5-minute systemd timer — had encountered a production emergency. All GPU workers were dead, 59 proof tasks were queued, and the agent's rate limiter was blocking all launch attempts. The agent correctly identified the crisis — its observation string read WORKERS DEAD — but it was powerless to act because the rate limiter enforced a 15-minute cooldown window between launches ([msg 4822]).

The assistant diagnosed this as a two-pronged failure: the rate limiter needed an emergency bypass, and the agent needed a way to signal that bypass. The solution was elegant and minimal: add an emergency boolean parameter to the launch_instance tool, and modify the Go backend to skip rate limit checks when emergency=true ([msg 4824] through [msg 4831]). The tool description was written to guide the LLM: "Set to true ONLY when workers_dead=true" ([msg 4837]).

The code was built, deployed, and the agent was triggered. The system appeared ready to handle the crisis.

The Moment of Discovery

Message [msg 4839] shows the deployment and the first agent run under the new logic. The agent log reveals the LLM's tool call:

launch_instance({"offer_id":32610028,"reason":"Emergency scale-up: Workers dead, need 500 p/h. Launching RTX 5090 (machine_id 8176)."})

The LLM described the situation as an emergency in the reason field, but it did not pass emergency=true. The parameter defaulted to false, and the rate limiter blocked the launch.

This is the critical observation in [msg 4840]. The assistant reads the log, spots the missing parameter, and immediately recognizes the failure mode. The LLM understood the context — it wrote "Emergency scale-up" in the reason — but it did not translate that understanding into the correct tool parameter. The instruction was present in the tool description, visible to the model, yet it was not followed.

Why Optional Booleans Fail

The assistant's diagnosis is worth examining in detail. The tool description said "Set to true ONLY when workers_dead=true" — a clear instruction by any human standard. Yet the model ignored it. Why?

The assistant's immediate conclusion is that the model "needs stronger nudging." This is a correct instinct, but the deeper issue is more subtle. Research into LLM tool-calling behavior — which the assistant would later commission in [msg 4842] through four parallel research tasks — reveals that optional boolean parameters are systematically unreliable for models in the 70B–122B parameter range. When a parameter is optional with a default value, models tend to skip it unless the instruction to use it is extraordinarily salient. The parameter is effectively invisible to the model's decision-making process, even when the model correctly identifies the situation that should trigger it.

The assistant's proposed fix — strengthening the system prompt — is the first step. But the deeper lesson, which the research in [msg 4844] would later confirm, is that the parameter should never have been optional in the first place. The SOTA recommendation is to use required enum parameters (launch_priority: "normal" | "emergency") instead of optional booleans. A required parameter cannot be skipped; the model must choose a value, and the choice forces the reasoning that the optional boolean allowed the model to avoid.

The Thinking Process Visible in the Message

The assistant's reasoning in [msg 4840] reveals a tight feedback loop between deployment and observation. The sequence is:

Deploy the fix (msg 4838–4839): Build, push, trigger the agent.
Observe the result (msg 4839 log): The LLM's tool call is captured in the journal.
Compare against expectation (msg 4840): The tool call lacks emergency=true.
Diagnose the gap: The tool description was present but insufficient.
Formulate the fix: Strengthen the system prompt. This is not a debugging cycle driven by errors or crashes — there is no exception, no stack trace, no HTTP 500. The failure is entirely behavioral: the LLM did what it thought was right, but it was not what the system needed. This is the unique challenge of building LLM-powered agents. Traditional software fails with clear signals; LLM agents fail by doing the wrong thing confidently and correctly.

Assumptions Tested and Broken

The message reveals several assumptions that the assistant held, and that were proven incorrect:

Assumption 1: Tool descriptions are sufficient instruction channels. The assistant wrote a clear description for the emergency parameter and assumed the LLM would follow it. The model read the description — it was in the prompt — but it did not act on it. Tool descriptions are not instructions; they are metadata. Models treat them as reference information, not as behavioral directives.

Assumption 2: The LLM would connect the observation to the parameter. The model saw WORKERS DEAD in its observation string and wrote "Emergency scale-up" in the reason field. It understood the situation. But understanding did not translate to action. The cognitive link between "this is an emergency" and "set emergency=true" was not made.

Assumption 3: An optional boolean with a clear trigger condition would be used correctly. The condition was simple: workers_dead=true. The model had that information. Yet the optional nature of the parameter allowed the model to skip it without any apparent reasoning cost.

Input and Output Knowledge

The input knowledge required to understand this message is substantial. One must grasp the agent architecture (Python agent calling a Go API that manages vast.ai instances), the rate limiter mechanism (15-minute cooldown between launches), the emergency bypass logic (skipping rate checks when emergency=true), and the deployment pipeline (build Go binary, scp to management host, restart systemd service). Without this context, the message reads as a trivial observation about a missing parameter.

The output knowledge created by this message is more profound. It establishes a concrete data point about LLM tool-calling reliability in production. It demonstrates that optional boolean parameters are unreliable for critical control flow in autonomous agents. It validates the need for redundant instruction placement — the same rule must appear in the tool description AND the system prompt AND potentially in the observation string. Most importantly, it seeds the insight that would drive the next major architectural evolution: the replacement of emergency: bool with launch_priority: enum, the reordering of the system prompt to place critical rules at the end (leveraging recency bias), and the addition of structured reasoning formats that force the model to articulate its decision before making tool calls.

The Broader Significance

Message [msg 4840] is a microcosm of the challenge of building reliable LLM agents. The assistant did everything right by conventional software engineering standards: the feature was minimal, the deployment was clean, the verification was immediate. Yet the system failed because the LLM did not follow instructions that were clearly present in its context window.

This is not a bug in the traditional sense. There is no code to fix, no race condition to resolve, no memory leak to patch. The fix is purely in the prompt engineering — in how instructions are phrased, where they are placed, and what structural constraints (required enums vs. optional booleans) are imposed on the model's output. The assistant's recognition of this, and its immediate pivot to prompt-level intervention, marks the transition from treating the LLM as a conventional API to treating it as a cognitive system with its own patterns of attention and forgetfulness.

The research that follows in [msg 4842] through [msg 4844] — four parallel sub-agents investigating SOTA prompting, tool definitions, context management, and event triggering — is a direct consequence of this moment. The assistant learned that tool descriptions alone are insufficient, and that the entire prompt architecture must be redesigned around how models actually process information, not how developers wish they would.