When Tool Calls Break: Debugging Parser Mismatches in Multi-Model Inference Stacks

Introduction

In the middle of an intense session tuning multi-step speculation parameters for Qwen3.5-122B across Blackwell GPUs — having just benchmarked steps=4 delivering 277 tok/s single-request throughput, more than double the baseline — the conversation takes an abrupt pivot. The user pastes a raw session log from inside the model itself, showing a cascade of failed tool calls:

⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: 
Text: {"filePath": "/tmp/lunal/lunal-research-notes.md".
Error message: JSON Parse error: Expected '}']

The user's closing question — "Are we using the correct tool call parser?" — lands like a diagnostic grenade. Message 6551 is the assistant's response, and it is a masterclass in structured debugging under uncertainty. This article examines that single message in depth: why it was written, the reasoning it encodes, the assumptions it makes, and the knowledge it both consumes and produces.

The Subject Message

Here is the message in full:

Good catch. The tool call parser is set to qwen3_coder but the model is generating malformed JSON for tool calls — the content parameter is missing or the JSON is truncated. This could be a parser mismatch issue.

>

Let me check what tool call parsers are available and whether qwen3 (without _coder) might be more appropriate for this model: [task] {"description":"Check tool call parsers","prompt":"On the remote server (ssh root@10.1.230.174), search the SGLang codebase at /root/sglang-main/ for:\n\n1. What tool call parsers are available? Search for where --tool-call-parser choices are defined in server_args.py.\n\n2. Find the implementa... task_id: ses_32b0cca6effe6dBCrJGjQa7UPA (for resuming to continue this task if needed)

The message is deceptively brief. It contains three distinct moves: an acknowledgment, a hypothesis, and a concrete investigative action. Each deserves scrutiny.

Why This Message Was Written: The Context and Motivation

To understand the motivation, we must understand what preceded it. The session had been entirely focused on infrastructure deployment and speculation tuning. The user had been iterating on speculative_num_steps values — 1, 2, 3, 4, 5, and 10 — benchmarking each, crashing servers, cleaning up zombie processes, and restarting systemd services. This is heavy systems engineering work: managing GPU memory, tuning KV cache allocation, wrestling with OOM conditions.

Then, without transition, the user drops a completely different kind of problem into the conversation. The session log shows a model (presumably the same Qwen3.5-122B they've been deploying) attempting to use tools — WebFetch, exa_web_search_exa, and write — and failing. The write tool fails twice with the same JSON parsing error. The model's own "Thinking" blocks reveal it understands the error ("my previous attempt had a JSON parsing error due to missing 'content' parameter") but then repeats the same mistake.

This is a fundamentally different class of problem from the speculation tuning that preceded it. It's not about throughput or memory pressure. It's about the semantic layer: how the model formats tool calls, how the server parses them, and how the client executes them. The assistant must context-switch instantly from systems engineering to language model alignment debugging.

The message is written because the user has flagged a potential configuration error — the wrong --tool-call-parser argument — and the assistant needs to validate or refute that hypothesis before the user wastes time debugging the wrong layer.

The Reasoning Process: A Three-Stage Diagnostic

The assistant's reasoning, visible in the message's structure, follows a clean diagnostic arc.

Stage 1: Observation and pattern recognition. The assistant immediately identifies the symptom: "the model is generating malformed JSON for tool calls — the content parameter is missing or the JSON is truncated." This is not a guess; it is a precise reading of the error message. The JSON parser expects an object with filePath and content keys, but receives only filePath. The assistant recognizes this as a parser-side failure, not a model-side hallucination.

Stage 2: Hypothesis formation. The assistant proposes that the qwen3_coder parser might be incorrect for this model, and that qwen3 (without the _coder suffix) might be more appropriate. This is a specific, testable hypothesis. It reflects deep knowledge of SGLang's architecture: different model families use different tool call parsers, and selecting the wrong one can cause exactly this kind of structured output failure.

Stage 3: Investigative action. Rather than speculating further, the assistant spawns a task subagent to search the SGLang codebase. The task prompt asks two precise questions: what parsers are available (defined in server_args.py), and how the qwen3_coder parser is implemented. This is the right move — it converts a hypothesis into a research question and dispatches it to a parallel reasoning process.

Assumptions Embedded in the Message

Every diagnostic rests on assumptions, and this message is no exception.

The first assumption is that the problem is server-side rather than client-side. The assistant assumes that the --tool-call-parser argument passed to SGLang determines how tool calls are parsed, and that changing it could fix the issue. This is correct in the SGLang architecture, but it's worth noting that the error could also originate from the client agent framework (e.g., OpenCode or Cursor) that receives the parsed tool call and attempts to re-serialize it. The assistant implicitly privileges the server-side explanation.

The second assumption is that the qwen3_coder and qwen3 parsers are meaningfully different. The assistant's question — "whether qwen3 (without _coder) might be more appropriate" — assumes that the _coder variant is designed for a specific model variant (perhaps Qwen3-Coder) and that the base Qwen3.5 model might need the non-coder parser. This is a reasonable assumption given naming conventions in the SGLang codebase.

The third assumption is that the model's tool call format is consistent with what the parser expects. The user's session log shows the model outputting tool calls in a format that includes ⚙ emoji prefixes and [numResults=10, query=...] style parameter syntax. The assistant assumes this format should match one of SGLang's built-in parsers. If neither qwen3_coder nor qwen3 works, the problem might be deeper — perhaps the model is using a custom tool format that no built-in parser handles.

Input Knowledge Required to Understand This Message

To fully grasp what is happening here, a reader needs several layers of context.

First, they need to understand the SGLang inference server architecture, particularly the --tool-call-parser flag. SGLang supports multiple parsers for different model families (Hermes, Mistral, Qwen, Llama, etc.), each designed to handle the specific tool call format that model was trained on. The qwen3_coder parser, for instance, expects an XML-ish format with <function=name> and <parameter=name>value</parameter> tags.

Second, they need to understand the Qwen3.5 model family. Qwen3.5-122B-A10B is a Mixture-of-Experts model with 122B total parameters and 10B active parameters per token. It was trained with a specific tool call format, and using the wrong parser means the server might fail to extract tool call arguments correctly.

Third, they need to understand the concept of "streaming parsers" in LLM serving. When a model generates tokens incrementally, the parser must operate on partial output — it sees the beginning of a tool call before the arguments are complete. The qwen3_coder streaming parser is described as "a complex state machine" (as the assistant notes in the follow-up message 6552), and edge cases in this state machine could cause truncation or malformed parsing.

Fourth, they need the immediate conversational context: the assistant had just finished deploying Qwen3.5-122B with MTP speculation, and the user's session log represents a separate interaction with that deployed model. The tool call failures are happening in a different agent session that is consuming the model's output.

Output Knowledge Created by This Message

This message produces several forms of knowledge.

The most immediate output is the task subagent result (visible in the truncated <task_result> block). The subagent searches the SGLang codebase and returns a comprehensive analysis of available parsers, their detector classes, and their implementation details. This knowledge is then used in the follow-up message (msg 6552) to confirm that qwen3_coder is actually the correct parser for this model.

The message also produces a refined diagnostic framework. By framing the problem as a potential parser mismatch, the assistant creates a clear decision tree: if the parser is wrong, fix the flag; if the parser is correct, look elsewhere (client-side serialization, streaming edge cases, content truncation). This framework is itself valuable knowledge — it teaches the user how to approach similar debugging scenarios in the future.

Perhaps most importantly, the message produces a corrected understanding of where the bug lives. The follow-up message (msg 6552) reveals that the parser choice is actually correct, and the real issue is client-side: "the downstream client receives the parsed tool call and then tries to re-serialize/parse the arguments as JSON." This is a significant insight that shifts the debugging focus from the server to the client agent framework.

The Task Subagent: A Deliberate Architectural Choice

The most interesting structural decision in this message is the use of a task subagent. The assistant could have run a simple bash command to grep the SGLang codebase, or it could have reasoned from its own knowledge. Instead, it chose to spawn a parallel reasoning process.

This is not an accident. The task tool in opencode creates a subagent that runs its own multi-round conversation, completely independent of the parent session. The parent blocks until the subagent completes. This means the assistant is consciously trading immediacy for depth: instead of a quick grep that returns raw code, it asks the subagent to "compile a comprehensive analysis." The subagent will read files, synthesize findings, and return structured knowledge.

The choice reflects an understanding that the tool call parser issue is nuanced. A simple grep for --tool-call-parser would reveal the list of parsers, but understanding which parser is correct for Qwen3.5-122B requires reading the chat template, understanding the model's training data format, and cross-referencing with the parser implementations. That's a reasoning task, not a lookup task.

The task prompt itself is carefully scoped. It asks two specific questions and includes a note to "check the chat template of the model" — showing that the assistant anticipates the need to verify not just what parsers exist, but which one matches the model's actual output format.

Mistakes and Incorrect Assumptions

The message's primary assumption — that the qwen3_coder parser might be wrong — turns out to be incorrect. In the follow-up message (msg 6552), the assistant concludes: "The qwen3_coder parser is correct for this model. The model's chat template explicitly uses the XML-ish format, which is exactly what qwen3_coder expects."

This is not a failure of reasoning. It is the natural outcome of a well-formed hypothesis being tested against evidence. The assistant made a reasonable guess based on the symptom (malformed JSON) and then immediately set out to verify it. When the evidence contradicted the hypothesis, the assistant updated its model and produced a more accurate diagnosis (client-side serialization issue).

The more subtle mistake is in the initial framing. The assistant says "the model is generating malformed JSON" — but the follow-up reveals that the model is actually generating correct XML tool calls (which the qwen3_coder parser handles correctly). The JSON parsing error occurs downstream, when the client agent tries to execute the tool. The assistant's initial framing conflates the model's output format with the client's parsing format, which are two different layers.

This is a forgivable error given the information available. The user's session log shows the error message from the tool execution layer, not from the model's raw output. Without seeing the raw model response, it's natural to assume the model itself produced bad JSON.

Broader Significance

Message 6551 represents a critical skill in modern AI infrastructure debugging: the ability to rapidly context-switch between systems engineering and language model alignment. The assistant goes from tuning CUDA graph capture parameters to analyzing tool call parser state machines in a single turn, without losing coherence.

The message also illustrates a deeper truth about multi-model inference stacks. When you deploy a model through a serving framework (SGLang), which is consumed by an agent framework (OpenCode), which calls tools through an execution layer — there are at least three independent systems that must agree on the format of tool calls. A bug at any layer looks like a bug at every layer. The assistant's diagnostic approach — isolate the layer, test the hypothesis, then pivot — is the only reliable way to navigate this complexity.

The message is, in essence, a demonstration of disciplined debugging under uncertainty. It acknowledges the observation, formulates a hypothesis, dispatches investigation, and prepares to update its understanding based on evidence. That is the scientific method applied to inference infrastructure, and it is the reason this seemingly small message deserves careful study.