The Wonky Transcript: Diagnosing Tool-Calling Failure in a Self-Hosted Reasoning Model

Introduction

In the life cycle of any serious machine learning deployment, there comes a moment when the infrastructure is complete, the benchmarks are green, and the monitoring dashboards are glowing — and then a human actually tries to use the model for something real. That moment, captured in a single user message within a sprawling opencode coding session, is the subject of this article. The message at index 12745 is a deceptively simple bug report: the user pastes a transcript of their self-hosted DeepSeek-V4-Flash model behaving erratically in an agent harness, compares it against the same model running on a third-party inference provider, and asks a pointed question: what went wrong?

What makes this message extraordinary is not the problem itself — tool-calling failures in self-hosted LLMs are common — but the depth of diagnostic information packed into a few paragraphs. The user provides raw transcripts, side-by-side comparisons, self-identified hypotheses, and even timing data from the third-party provider's reasoning traces. It is, in effect, a complete case study in why reasoning models fail when deployed without their reasoning capability enabled. This article unpacks every layer of that message: the context that produced it, the assumptions it challenges, the knowledge it creates, and the cascade of fixes it would ultimately trigger.

The Message in Context: A Deployment Comes Alive

To understand why message 12745 was written, one must first understand what preceded it. The conversation leading up to this point (segments 63–68 of the opencode session) documents an extraordinary engineering effort: deploying DeepSeek-V4-Flash — a massive Mixture-of-Experts reasoning model with 8-bit floating-point KV cache and NVFP4 quantization — onto a cluster of 8× RTX PRO 6000 Blackwell GPUs (sm_120 architecture). The assistant had, over the course of many rounds, designed custom MMA sparse-MLA decode kernels in Triton, flipped FP32 indexer operations to bf16 tensor-core operations, discovered and fixed an O(max_context) bottleneck that was causing the indexer to compute scores over 262,208 positions every decode step even when the actual context was only 512 tokens, deployed prefill-decode disaggregation across two NUMA domains with systemd services, set up a full Prometheus and Grafana monitoring stack with a 17-panel KV-cache dashboard, and written a comprehensive engineering report.

The very message immediately before this one (msg 12744) is the assistant's triumphant summary: "Done — full Prometheus + Grafana monitoring stack is deployed and verified." The KV gauge is moving, the metrics are flowing, the systemd services are enabled at boot. The deployment is, from an infrastructure perspective, complete.

Then the user tries to use it.

Message 12745 is the user returning from that first real test with a transcript in hand. The model has been plugged into an "agent harness" — presumably a framework that gives the model access to tools like bash, write, read, and grep — and asked to write a "helloword html landing page with js animation." What follows is a train wreck: the model produces a truncated HTML page, then seems to forget it has tools, asks clarifying questions instead of acting, hallucinates a non-existent tool called run, gets confused about its working directory, and eventually produces a mediocre result after a long and painful interaction. The user's diagnosis is characteristically sharp: "confused about tools, barely can do tool calling, confused about env etc."

Anatomy of a Wonky Transcript

The transcript the user provides is worth quoting in full, because every line tells a different story about what's broken. The first model output begins promisingly — "Here is an HTML landing page with interactive JavaScript animations" — and then produces a truncated HTML document that cuts off mid-style. The page has a globe with radial gradients, a glassmorphism card, and the beginning of a repeating-linear-gradient on a ::before pseudo-element. But it's interrupted mid-declaration: rgba(255 with no closing parenthesis, no body, no script tag. The model was clearly generating a complete page but was cut off, either by a token limit or by the harness itself. This is the first symptom: the model cannot reliably produce long-form structured output within the constraints of the agent harness.

What follows is far more concerning. The model's next response, to what must have been a follow-up prompt from the harness, is: "can you call tools?" followed by a list of available tools. This is bizarre behavior for a model that has just been asked to write an HTML file. Instead of calling write_file or bash to create the file, the model is asking the user whether it can use tools. This suggests the model does not understand that it is expected to use tools — it treats the tool list as informational rather than actionable. The model is in "chat mode," not "agent mode."

The pattern continues. Asked "can you write to file?", the model responds with a tutorial about its file-writing capabilities, ending with "What are you looking to create/file? 🚀" — a question that should have been unnecessary given the original request. The model is recursively asking for clarification instead of acting. When the user says "do that," the model asks for more context: "I need a bit more context – what would you like me to write to a file?" At this point, the model has been told three times to write an HTML landing page, and it is still asking what to do.

When the user finally says "Write the html page to file," the model begins to act — but its actions reveal profound confusion. It runs ls -la /home/user/ to check the environment. It tries to call a tool called run, which does not exist in the opencode tool set (the correct tool is bash). The harness returns an error: "Model tried to call unavailable tool 'run'. Available tools: bash, edit, exa_web_fetch_exa, exa_web_search_exa, glob, grep, invalid, question, read, skill, task, todowrite, webfetch, write." The model then pivots to reading files in /Users/ritwik/expense-tracker-2 — a completely irrelevant path from a different project — before finally producing a helloworld.html file.

The final output is a competent but unremarkable HTML page with a particle network animation. It works, but it lacks the polish of the first truncated attempt (which had a globe, glassmorphism card, and gradient text). The model has degraded from "can produce a beautiful page but gets interrupted" to "can produce a mediocre page after much confusion."

The Side-by-Side Revelation

The most powerful part of message 12745 is the comparison. The user provides, in the same message, the output from a third-party inference provider running the same model with the same prompt. The difference is stark.

The third-party output is clean, complete, and confident. It produces a full HTML document with a particle system: 150 particles with position, velocity, opacity, and size; an animation loop with requestAnimationFrame; inter-particle distance calculations that draw connecting lines when particles are within 120 pixels of each other; a floating animation on the heading text; and a responsive canvas that resizes with the window. The code is well-structured, uses modern CSS features like clamp() for responsive typography and background-clip: text for gradient text, and includes thoughtful touches like letter-spacing and opacity gradients. The model's internal reasoning is visible: "Thought: 366ms" before the first response, "Thought: 73ms" before the final confirmation. The model thought about what to do, then did it.

The contrast could not be clearer. The same model, the same prompt, two radically different behaviors. On the third-party provider, the model is a capable agent that produces correct output efficiently. On the self-hosted deployment, the model is confused, hesitant, and error-prone. The user's annotation captures the essence: "confused about tools, barely can do tool calling, confused about env etc."

But the user also provides the crucial clue: "(thinking set to max; I couldn't set effort on our model btw)." The third-party provider ran the model with thinking/reasoning enabled at maximum effort. The self-hosted deployment could not set this parameter. This single sentence is the key to the entire mystery.

The User's Diagnostic Framework

The user does not simply present the problem — they present a diagnosis. In a few lines of commentary, they identify three potential root causes:

Tool-call parser mismatch: "Are we correctly setting up toolcall parser matching the model?" The user knows that DeepSeek-V4 uses a specific format for tool calls, and that the harness must parse this format correctly. If the parser doesn't match what the model was trained on, the model's tool calls will be misread or ignored.
Quantization degradation: "Are we using some sketchy quant (I thought no?)" The deployment uses NVFP4 quantization — 4-bit floating-point weights. The user is aware that aggressive quantization can degrade model quality, particularly for complex tasks like multi-turn tool use. They believe they're not using a "sketchy" quant, but they're not certain.
Thinking/reasoning disabled: The user notes that the third-party provider had thinking set to max, while their deployment couldn't set it. This is presented almost as an afterthought, but it is the most important observation. The user's diagnostic framework is remarkably complete for a first report. They have identified the three dimensions along which the deployment differs from the reference implementation: tool format, quantization level, and reasoning configuration. They have not yet prioritized these factors — that will be the assistant's job in the following messages — but they have laid out the search space with precision.

Assumptions and Blind Spots

Message 12745 exposes several assumptions that had been made during the deployment process, some of which turn out to be incorrect or incomplete.

Assumption 1: The chat template works correctly. The deployment used --chat-template tool_chat_template_deepseekv32.jinja — a template designed for DeepSeek-V3.2, not DeepSeek-V4-Flash. The assumption was that the V3.2 template was close enough to work, or that the model would adapt. In reality, the V4 model has its own specific chat template that handles system prompts, thinking triggers, and tool injection differently. Using the wrong template means the model receives prompts in a format it wasn't trained on, which can cause it to misinterpret instructions, ignore tool definitions, or fail to enter "thinking mode."

Assumption 2: The tool-call parser is correct. The deployment used --tool-call-parser deepseekv4, which should be correct for the model. But the parser is only half the equation — the model must also emit tool calls in the format the parser expects. If the chat template doesn't instruct the model to use that format, or if thinking is disabled and the model can't reason about which format to use, the parser will receive malformed or missing tool calls.

Assumption 3: The model works without thinking enabled. This is the most consequential assumption. DeepSeek-V4-Flash is a reasoning model — it was trained to produce chains of thought before answering. Without thinking enabled, the model is operating outside its training distribution. It's like asking a mathematician to solve problems without showing their work: they can still produce answers, but the quality and reliability suffer dramatically. The deployment defaulted thinking to off (the deepseekv32 template has default_enabled=False), which means every agentic interaction starts with the model's reasoning capability disabled.

Assumption 4: The harness configuration is correct. The user's transcript shows the model trying to call a tool called run — a tool that doesn't exist in the opencode environment. This suggests the model was either given incorrect tool definitions, or it hallucinated the tool name because it couldn't reason about which tools were actually available. The third-party provider's clean output suggests the model can use tools correctly when properly configured.

Assumption 5: Quantization is not a factor. The user says "I thought no" to the question of sketchy quantization, but the transcript raises doubts. NVFP4 is a 4-bit format that preserves more dynamic range than INT4 but still represents a significant compression of the model's weights. While the model may pass benchmark evaluations in this format, complex agentic behavior — which requires nuanced understanding of tool descriptions, environment state, and multi-step planning — may be more sensitive to quantization than single-turn text generation.

Input Knowledge Required to Understand This Message

To fully grasp message 12745, a reader needs substantial context from the preceding conversation. This message does not exist in isolation; it is the culmination of dozens of earlier messages that established the deployment's architecture and configuration.

The deployment architecture: The reader must know that DeepSeek-V4-Flash is deployed on 8× RTX PRO 6000 Blackwell GPUs with prefill-decode disaggregation, custom Triton MMA kernels, and NVFP4 quantization. They must understand that the model runs via SGLang with specific flags for tensor parallelism, context length, and memory management.

The chat template choice: The reader must know that the deployment uses tool_chat_template_deepseekv32.jinja as a proxy for the V4 template, and that this was a deliberate workaround because the official V4 template wasn't available at deployment time.

The thinking configuration: The reader must know that thinking is disabled by default in the deployed template, and that the user was unable to set thinking or reasoning_effort parameters through the harness.

The monitoring stack: The reader should understand that the deployment has full Prometheus/Grafana monitoring, which means the assistant can query metrics to correlate model behavior with system state.

The prior optimization work: The reader should appreciate that the deployment has been through an intensive optimization campaign — custom kernels, NCCL tuning, PD disaggregation — which means the infrastructure is mature and the remaining issues are likely configuration or software-level rather than hardware or performance.

The third-party reference: The reader needs to understand that the comparison point is a production-grade inference provider with optimized serving infrastructure, which represents the "gold standard" for model behavior.

Output Knowledge Created by This Message

Message 12745 creates several pieces of critical knowledge that drive the subsequent conversation.

First, it establishes that the deployment has a functional problem, not just a performance problem. The earlier optimization work focused on throughput (tokens per second), latency (time to first token), and memory utilization (KV cache pressure). This message reveals that the model is producing wrong behavior — not just slow behavior. This is a different class of issue requiring different diagnostic tools.

Second, it provides a reproducible failure case. The transcript gives the assistant a concrete scenario to test against: "Write a helloword html landing page with js animation" in an agent harness with tool access. The assistant can now run this exact prompt against the deployment and compare the output to the transcript.

Third, it identifies the thinking/reasoning configuration as the most likely root cause. The user's observation that the third-party provider ran with "thinking set to max" while the self-hosted deployment couldn't set it is the single most valuable piece of diagnostic information in the message. It narrows the search space dramatically.

Fourth, it raises the quantization question. Even though the user says "I thought no" to sketchy quantization, the message puts NVFP4 on the table as a potential contributing factor. This will need to be investigated — perhaps by comparing FP8 and NVFP4 behavior on the same agentic task.

Fifth, it creates a benchmark for tool-calling quality. The transcript documents specific failure modes: hallucinated tool names (run), recursive clarification loops, irrelevant file reads, truncated output. These become measurable metrics for evaluating fixes. A successful fix should eliminate all of these failure modes.

Sixth, it establishes a methodology for future debugging: side-by-side comparison with a reference implementation. The user's approach of running the same prompt on a third-party provider and comparing outputs is a powerful diagnostic technique that the assistant will use repeatedly in subsequent rounds.

The Thinking Process: What the User's Analysis Reveals

While message 12745 is a user message and does not contain explicit "reasoning" tags like the assistant's messages do, it reveals a sophisticated thinking process. The user has:

Observed the failure: The model produced wonky behavior in the agent harness.
Captured evidence: The user saved the full transcript, not just a summary.
Established a baseline: The user ran the same prompt on a third-party provider to get a reference output.
Identified differences: The user noted that the third-party provider had thinking enabled and their deployment didn't.
Formulated hypotheses: The user proposed three potential causes (tool-call parser, quantization, thinking).
Presented the evidence for analysis: The user shared everything with the assistant rather than jumping to conclusions. This is a model of how to report a bug in a complex system. The user doesn't say "fix it" — they say "here's what happened, here's what I think might be wrong, what do you think?" This collaborative diagnostic approach is what makes the subsequent fix cycle so productive. The user's note about thinking is particularly insightful. They could have simply said "the model is broken." Instead, they noticed a specific configuration difference between their deployment and the reference, and they flagged it. This is the kind of observation that comes from deep familiarity with how these models work — the user knows that reasoning models need reasoning enabled, and they recognized that the absence of thinking traces in their deployment's output was a red flag.

The Road to Resolution

Message 12745 sets in motion a cascade of diagnostic and remedial actions that will occupy the next several rounds of the conversation. The assistant's response (msg 12746) immediately latches onto the thinking clue: "DeepSeek-V4 is a reasoning model — the 3rd-party ran it with thinking maxed (you can see its Thought: traces), while our endpoint defaults thinking OFF."

The assistant identifies three likely culprits in order of impact:

Thinking being OFF (the biggest and easiest fix)
Using a V3.2 template proxy instead of the official V4 template
NVFP4 quantization potentially degrading reasoning The assistant's plan is methodical: test thinking enabled vs. disabled on an agentic prompt, fetch the official V4 chat template from Hugging Face, and compare the tool-call format. The fix will involve patching the chat template to enable thinking by default, ensuring the tool-call parser matches the model's output format, and potentially adjusting the quantization configuration. The broader lesson is that deploying a reasoning model requires more than just serving infrastructure — it requires understanding the model's interaction paradigm. A reasoning model without reasoning enabled is like a car without an engine: it may look functional, but it won't move. The user's message 12745 is the moment this truth becomes undeniable.

Broader Lessons for Self-Hosted Reasoning Models

Message 12745 illustrates several principles that apply to any deployment of a reasoning model in an agentic context.

Reasoning is not optional for reasoning models. This seems obvious in retrospect, but it's easy to overlook during deployment. The focus on throughput, latency, and memory optimization can obscure the fact that the model's fundamental capability — its ability to reason through complex tasks — depends on having thinking enabled. Disabling thinking to save tokens or latency is like removing the spark plugs to improve fuel efficiency: you'll use less fuel, but the car won't run.

Chat templates are not interchangeable. Using a V3.2 template for a V4 model seems like a minor approximation, but it can have catastrophic effects on model behavior. The template controls how the system prompt, tool definitions, user messages, and thinking triggers are formatted. If the format doesn't match what the model was trained on, the model will misinterpret the entire interaction.

Side-by-side comparison is the most powerful debugging tool. The user's decision to run the same prompt on a third-party provider and compare outputs is what made the diagnosis possible. Without that reference, the wonky behavior might have been attributed to any number of causes — hardware issues, network latency, harness bugs. The comparison instantly ruled out model-level problems and focused attention on deployment configuration.

Agent harnesses expose configuration issues that benchmarks miss. The model had passed throughput and latency benchmarks. It had served single-turn chat requests correctly. But the agent harness — with its multi-turn tool use, environment interaction, and reasoning requirements — revealed configuration problems that simpler tests couldn't detect. This is why real-world testing is irreplaceable.

The user's diagnostic skill matters as much as the assistant's technical skill. Message 12745 is effective because the user is a skilled diagnostician. They captured evidence, established baselines, identified differences, and formulated hypotheses. The assistant's technical ability to fix the problem would have been useless without the user's ability to find the problem. This partnership — user as diagnostician, assistant as technician — is the core dynamic of the opencode session.

Conclusion

Message 12745 is, on its surface, a simple bug report: a self-hosted DeepSeek-V4-Flash model behaves erratically in an agent harness, while the same model on a third-party provider works correctly. But beneath that surface lies a rich case study in the challenges of deploying reasoning models, the importance of configuration fidelity, and the art of collaborative debugging.

The message succeeds because of what the user brings to it: a keen eye for anomalous behavior, the discipline to capture complete evidence, the initiative to establish a reference baseline, and the insight to identify the most likely root cause. The user's observation that "thinking set to max; I couldn't set effort on our model" is the key that unlocks the entire mystery.

For the assistant, message 12745 is a turning point. The optimization work that preceded it — the custom kernels, the NCCL tuning, the PD disaggregation, the monitoring stack — was necessary but not sufficient. The deployment needed one more thing: the correct configuration for the model's reasoning capability. The user's message provides the evidence and the direction; the assistant's subsequent messages will provide the implementation.

For the reader, message 12745 is a reminder that deploying a large language model is never just about infrastructure. It's about understanding the model as a cognitive system — what it needs to function, how it processes instructions, and what happens when its core capabilities are accidentally disabled. The wonky transcript is not a bug in the model; it's a bug in the deployment. And fixing it requires not just technical skill, but the kind of diagnostic thinking that the user demonstrates in this single, remarkable message.