The Reasoning Leak: How a Multi-Turn Context-Loss Bug Was Traced to Prior-Thinking Contamination

Introduction

In the high-stakes world of deploying large language models for production agent systems, few bugs are as insidious as the silent context-loss failure. The model appears to work—it generates coherent text, follows instructions, and completes tasks—but over multiple turns of conversation, it gradually loses the thread. It forgets what was said two turns ago. It acts as though the user's latest request is the very first message. The symptoms are subtle at first, then catastrophic: agents fail to complete multi-step tasks, tools are called with wrong parameters, and the entire conversation degrades into a frustrating game of "I already told you this."

This article examines a single message in an opencode coding session—message index 12836—where the assistant, after hours of chasing performance patches, kernel bugs, and numerical stability issues, finally discovers the true root cause of a multi-turn context-loss failure in a production deployment of DeepSeek-V4-Flash (DSV4) on Blackwell GPUs. The discovery is deceptively simple: the model's prior-turn reasoning (chain-of-thought) was being fed back into the prompt as conversation history, violating the model's training distribution and causing it to lose context across turns.

But the path to this discovery reveals far more than a simple encoding bug. It exposes the tension between performance optimization and numerical correctness, the dangers of incomplete validation, and the critical importance of understanding how a model was trained to consume its own output.

The Setup: A Production Deployment Under Stress

To understand message 12836, we must first understand what led to it. The broader session (segment 69 of the conversation) documents a production deployment of DeepSeek-V4-Flash on a machine with eight RTX PRO 6000 Blackwell GPUs. The assistant had spent considerable effort optimizing the deployment: custom MMA split-K attention kernels, a Triton-based indexer with page early-exit logic, bf16 GEMM replacements for fp32 operations, and MoE routing optimizations. These patches were necessary to achieve acceptable throughput—the stock SGLang deployment was bottlenecked by fallback kernels that didn't support the Blackwell sm_120 architecture.

The deployment used a prefill-decode disaggregation architecture, where separate systemd services handled prompt processing (prefill) and token generation (decode). Prometheus and Grafana monitoring had been set up. The system appeared stable. Single-turn coherence tests passed. An 8K-token "needle test" showed no degradation.

Then the user reported a problem: their agent harness (opencode) consistently lost context on long conversations. The model would act as if prior turns never happened. The specific example was a tic-tac-toe request: the user asked the model to "write a tic tac toe HTML page," the model began generating HTML, the user then said "to a file" (meaning "save it to a file"), and the model responded with something like "this is the very first message"—as though no prior conversation existed.

The Initial Misdiagnosis

The assistant's first hypothesis (in message 12831) was that the problem was temperature-induced repetition collapse. The harness was sending temperature=0.2, which is much closer to greedy sampling than the default 0.6. At that temperature, DeepSeek-V4 is prone to repetition, especially when the context already contains repetitive content. The assistant pointed to logs showing a ProofShare/SNARK proving agent that ran every five minutes, sending the same status message repeatedly, and the model's responses degenerating into repetitive boilerplate.

The user pushed back forcefully (message 12832): "This conversation is whatever, it's an automated harness that will recover eventually; The issue far far more obvious in e.g. the opencode conversation." The user wanted the assistant to look past the automated harness logs and examine the actual opencode conversation where the tic-tac-toe failure occurred.

This pushback was the turning point. The assistant pivoted from theorizing about temperature and repetition to doing actual forensic analysis of the request logs.

The Forensic Investigation

Messages 12833 through 12835 document a systematic investigation. The assistant searched the prefill server's request logs (which were enabled with --log-requests) for the specific tic-tac-toe exchange. It found two lines totaling 87KB of log data: a "Receive OpenAI" line containing the raw message array sent by the harness, and a "Finish" line containing the fully-templated prompt that was actually fed to the model.

In message 12835, the assistant wrote a Python script to parse these lines and extract the message structure. The output showed four messages in the array:

The Breakthrough: Message 12836

Message 12836 is where everything clicks into place. The assistant's reasoning block reveals a mind working through multiple competing hypotheses, discarding each one in turn, until the critical insight emerges.

Let me quote the key section of the assistant's reasoning:

"Found it. Look at the tail of the templated prompt actually fed to the model:

>

`` write a tic tac toe html page<think>The user wants me to create a tic-tac-toe HTML page. I'll build a complete, polished implementation...</think> `html ...partial html...rgba(255, 255, 255, 0.to a file<think> ``

>

The previous assistant turn's chain-of-thought (<think>…</think>) is being fed back into the prompt as history. Note message[2] as received (len 726) starts at \n`html — it has no think text — yet the templated prompt injects <think>The user wants me to create a tic-tac-toe…</think> before it. So the reasoning came from a separate field and our encoding is re-emitting prior-turn reasoning into context."

This is the "smoking gun" moment. The assistant then runs a Python script to confirm the mechanism:

--- msg[2] role=assistant keys=['role', 'content', 'reasoning_content']
      content: len=726  head='\n```html\n<!DOCTYPE html>\n...

The message has a reasoning_content field separate from content. The opencode harness correctly stores the assistant's prior reasoning and sends it back as a separate field. But the SGLang encoding layer then re-emits this reasoning_content as &lt;think&gt;...&lt;/think&gt; tags in the conversation history—even though the model's training data never included prior-turn reasoning in the prompt.

Why This Breaks Multi-Turn Conversations

The DeepSeek chat template is designed with a specific assumption: prior assistant turns in the conversation history should contain only the output (the visible response), not the reasoning (the chain-of-thought). The reasoning is generated fresh each turn and is not meant to be fed back in subsequent requests.

When the encoding layer re-injects prior reasoning into the prompt, several things go wrong:

  1. Out-of-distribution input: The model was trained on conversations where prior assistant turns contained only final output. Seeing &lt;think&gt; tags from previous turns in the prompt is outside its training distribution, causing it to process the context differently than intended.
  2. Context window pollution: At reasoning_effort=max, a single turn's reasoning can balloon to thousands of tokens. When these accumulate across multiple turns, they consume massive amounts of the context window—tokens that could have been used for actual conversation history.
  3. Attention dilution: The model's attention mechanism must now spread its attention across thousands of tokens of prior reasoning that it wasn't designed to process. This dilutes the attention paid to the actual conversation history, making it harder for the model to recall what was said.
  4. Structural confusion: The malformed assistant turn (ending mid-code with an EOS token right before the next user message) further breaks the model's ability to parse the conversation structure, causing it to lose track of which messages belong to which role. The assistant's reasoning captures this perfectly: "DeepSeek requires prior &lt;think&gt; to be stripped from history; keeping it is OOD and, at reasoning_effort=max (thousands of reasoning tokens/turn), it accumulates and destabilizes every multi-turn agent session."

The Thinking Process: A Case Study in Debugging Methodology

What makes message 12836 remarkable is not just the discovery itself, but the thinking process that led to it. The assistant's reasoning block reveals a sophisticated debugging methodology:

Hypothesis generation and淘汰: The assistant cycles through multiple hypotheses in rapid succession. First, it considers whether the chat template is rendering reasoning from a separate field (correct). Then it considers whether the malformed assistant turn (ending mid-code) is breaking the model's ability to parse conversation structure (plausible but not primary). Then it considers whether custom kernels are breaking prefill attention at chunk boundaries (possible but not supported by evidence). Finally, it arrives at the correct hypothesis: prior reasoning is leaking into the prompt.

Evidence-driven iteration: Each hypothesis is tested against the available evidence. The assistant doesn't just theorize—it runs Python scripts to examine the actual message structure, confirming that reasoning_content is a separate field and that the content itself does not contain &lt;think&gt; tags.

Awareness of confirmation bias: The assistant explicitly acknowledges its own shifting perspective: "I'm realizing the core issue: our chat encoding is feeding the previous assistant turn's reasoning block back into the prompt." This self-awareness is crucial in debugging—the willingness to abandon a previously held hypothesis (temperature-induced repetition) in favor of a better one.

Root cause vs. proximate cause: The assistant distinguishes between the proximate cause (the encoding layer re-emitting reasoning) and the root cause (the harness sending reasoning_content back, combined with the encoding layer not stripping it). This distinction matters because the fix could be applied at either level.

Assumptions and Mistakes

The debugging journey to message 12836 was not without missteps. Several assumptions proved incorrect:

The temperature assumption: The assistant initially blamed the low temperature setting (0.2) for causing repetition collapse. While low temperature does increase repetition probability, it wasn't the primary cause of the context-loss failure. The user's pushback was correct—the problem was structural, not parametric.

The kernel assumption: The assistant spent considerable effort auditing performance patches—the MMA split-K kernel, the Triton indexer, the bf16 GEMM replacements. These were red herrings for the context-loss bug. They might have introduced numerical approximations, but they weren't causing the model to forget prior turns.

The "coherency check" assumption: The earlier validation (a gzip-ratio-based coherency check) only caught repetition collapse, not context-fidelity. The assistant had declared the deployment stable based on a test that didn't actually test multi-turn recall.

The chunked-prefill assumption: The assistant hypothesized that the indexer's page early-exit logic might be breaking at chunk boundaries during prefill of long prompts. This was a sophisticated theory that turned out to be wrong, but it led to the productive realization that the real issue was in the encoding layer, not the attention kernel.

These mistakes are instructive. They show how even experienced engineers can be led astray by plausible-sounding hypotheses, and why direct evidence (the actual templated prompt) is always more valuable than indirect inference.

Input Knowledge Required

To fully understand message 12836, several pieces of knowledge are necessary:

DeepSeek chat template design: The DeepSeek model family uses a specific chat template where prior assistant reasoning (&lt;think&gt; blocks) is supposed to be stripped from history. This is documented in the model card and is a deliberate design choice based on how the model was trained.

OpenAI-compatible API conventions: The opencode harness follows OpenAI's API conventions, where assistant messages can include a reasoning_content field alongside content. This field is meant for streaming display of chain-of-thought, not for inclusion in subsequent prompts.

SGLang encoding architecture: SGLang's encoding layer processes the OpenAI-compatible message array and applies the chat template to produce the final prompt text. The encoding must correctly handle the reasoning_content field according to the model's template specification.

Prefill-decode disaggregation: The deployment uses separate services for prefill and decode. Request logging is enabled on the prefill service, which is where the evidence was found.

The specific deployment patches: The assistant had applied several performance patches to the SGLang codebase. Understanding which patches touched the encoding layer (versus the attention or MoE layers) was crucial for correctly attributing the bug.

Output Knowledge Created

Message 12836 creates several valuable outputs:

A confirmed root cause: The prior-turn reasoning leak is definitively identified as the cause of multi-turn context loss. This is supported by direct evidence from the templated prompt and the message structure.

A reproducible diagnostic method: The Python script that dumps message keys and checks for reasoning_content provides a reusable technique for diagnosing similar issues in any deployment.

A clear fix direction: The fix is straightforward—strip &lt;think&gt;...&lt;/think&gt; from previous assistant messages when building the conversation history, either in the encoding layer or by not sending reasoning_content back to the client.

A validation gap identified: The earlier single-turn coherence tests were insufficient. Proper multi-turn recall tests (needle tests at 10-30K tokens) are needed to validate the fix.

A broader lesson about performance optimization: The assistant's journey from kernel debugging to encoding bug highlights a critical principle: when optimizing ML systems, the most impactful bugs are often not where you're looking. Performance patches can introduce subtle correctness issues, but the most catastrophic failures can come from much simpler sources.

The Broader Significance

The reasoning leak bug documented in message 12836 is a cautionary tale about the gap between how models are trained and how they are deployed. DeepSeek-V4-Flash was trained on carefully curated conversations where reasoning was ephemeral—generated during inference, not preserved in history. But the deployment pipeline, designed for maximum compatibility with OpenAI-style APIs, preserved and re-injected that reasoning, creating an out-of-distribution input that the model couldn't handle.

This tension between training distribution and deployment reality is one of the most challenging problems in production ML. Every optimization, every API compatibility layer, every encoding decision risks creating a mismatch between what the model expects and what it receives. The most dangerous mismatches are the silent ones—where the model still generates plausible text, but its internal representations are subtly corrupted.

The fix itself is simple: strip prior reasoning from history. But discovering that this was the fix required hours of investigation, multiple false starts, and the willingness to question every assumption about where the bug might be. It required looking past the performance patches that seemed like the obvious culprit, past the temperature settings that seemed like the obvious explanation, and directly at the actual data flowing through the system.

Conclusion

Message 12836 represents a turning point in the debugging session. Before it, the assistant was chasing performance patches and kernel bugs. After it, the path forward was clear: fix the encoding layer, validate with proper multi-turn tests, and move on.

But the message is more than just a debugging milestone. It's a masterclass in systematic investigation under uncertainty. The assistant's reasoning process—cycling through hypotheses, testing each against evidence, discarding the ones that don't fit, and arriving at the correct explanation—is a model for how to approach complex system failures.

The reasoning leak bug also serves as a reminder that in ML systems, the most important component is not the kernel, the optimizer, or the architecture—it's the data. The model is a reflection of its training data, and any deployment that feeds it data outside that distribution is asking for trouble. The fix for the reasoning leak is simple, but the lesson is profound: understand how your model was trained, and feed it only what it expects to see.

In the end, the bug that broke multi-turn conversations wasn't a GPU kernel bug, a numerical precision issue, or a memory corruption. It was a data pipeline bug—a mismatch between what the model was trained on and what it was being fed. And that, perhaps, is the most important lesson of all.