The Reasoning Content Leak: How Prior Chain-of-Thought Poisoned a Multi-Turn Agent Deployment

Introduction

In the high-stakes world of deploying large language models for production agentic workloads, the difference between a working system and a broken one often comes down to the most mundane details of prompt construction. On June 18, 2026, an AI assistant debugging a production deployment of DeepSeek-V4-Flash on NVIDIA Blackwell GPUs made a pivotal discovery: the model's prior chain-of-thought reasoning was being silently fed back into subsequent conversation turns, creating an out-of-distribution input that progressively destabilized multi-turn interactions. This article examines message 12838 of the conversation, the moment when the assistant confirmed the mechanism of this "reasoning content leakage" and began tracing its root cause through the deployment's encoding pipeline.

The message sits at a critical inflection point in a debugging session that had already spanned dozens of rounds. The user—an engineer running an opencode agent harness against a custom SGLang deployment—had reported that their model consistently "lost context" on longer conversations. In the canonical example, the model was asked to create a tic-tac-toe HTML page, produced a partial response, and then on the follow-up "to a file" claimed it had no prior context—acting as if "this is the very first message." The assistant had initially blamed temperature settings and repetition collapse, but the user pushed back, demanding a review of the performance patches applied to the SGLang inference server.

The State of Play Before Message 12838

To understand the significance of message 12838, we need to trace the investigation that led to it. The debugging session had been running for many rounds, with the assistant cycling through several hypotheses:

Hypothesis 1: Temperature-induced repetition collapse. The assistant examined logs from an automated proof-agent harness and found that the model was producing repetitive boilerplate at temperature 0.2. The assistant initially argued this was the primary cause—low temperature plus pathological repeated input creating a self-reinforcing loop of degeneration. But the user dismissed this, pointing to the tic-tac-toe example as a cleaner failure case.

Hypothesis 2: Kernel-level numerical errors at long context. The assistant considered whether the custom MMA split-K decode kernels, Triton indexer, or bf16 GEMM patches were introducing subtle attention errors that compounded over long contexts. The single-turn coherence test had passed, but that test used only ~90 tokens of prompt, while the opencode requests were ~10K tokens just from system prompts and tool definitions.

Hypothesis 3: Chunked prefill boundary issues. The assistant theorized that the page early-exit logic in the Triton indexer might be mishandling chunk boundaries during prefill of long prompts.

Hypothesis 4 (the breakthrough): Reasoning content leaking into history. In message 12836, the assistant extracted the actual templated prompt from the prefill request logs and found the smoking gun: the model's prior chain-of-thought (<think>...</think>) was being injected into the conversation history. The raw OpenAI message array showed that message[2] (the assistant's prior turn) had content starting at \n`html with no think text—yet the templated prompt showed <think>The user wants me to create a tic-tac-toe HTML page...</think> prepended to it. The reasoning was coming from a separate reasoning_content field that the encoding pipeline was re-emitting into the prompt.

Message 12838: Confirming the Mechanism

Message 12838 is the assistant's response to this discovery. It opens with a reasoning block that crystallizes the finding:

I'm seeing how the system handles reasoning content—the opencode harness correctly stores the assistant's prior reasoning and sends it back as a separate field, which our encoding then renders back into the conversation history as <think> tags. The mechanism involves opencode following OpenAI conventions while our sglang encoding processes it further downstream.

This is the moment of synthesis. The assistant has connected three pieces of evidence:

  1. The opencode harness stores the assistant's reasoning_content (the chain-of-thought from extended thinking) as a separate field in the message object, following the OpenAI API convention for separating visible content from reasoning.
  2. When the harness sends the next request, it includes this reasoning_content field in the assistant message.
  3. The SGLang encoding pipeline (encoding_dsv4.py) reads this field and renders it back into the conversation history as <think>...</think> tags. The assistant then states the core claim:
The core issue is that DeepSeek models don't expect prior-turn reasoning to be fed back into the prompt—it's out-of-distribution and causes multi-turn instability.

This is a critical assertion about the model's training distribution. DeepSeek-V4-Flash was trained on conversations where prior reasoning blocks were stripped from history. When the model sees its own prior chain-of-thought re-injected into the prompt, it encounters an input pattern that differs from its training distribution—an out-of-distribution (OOD) input. The assistant argues that this OOD condition is what causes the model to "lose context" on longer conversations.

The Proposed Fix and Its Nuances

The assistant proposes a straightforward fix:

So regardless of whether this rendering behavior is from our patch or stock sglang, the fix is to strip reasoning content from historical assistant messages when building the prompt, keeping it only for the current turn being generated (though actually, reasoning is only generated fresh, not carried forward).

This is a clean, surgical intervention: strip reasoning_content from all assistant messages except the current turn being generated. The assistant correctly notes that reasoning is only generated fresh—it's never "carried forward" intentionally—so stripping it from history should have no negative side effects.

However, the assistant also reveals uncertainty about whether this is the primary cause or just one contributing factor:

I'm trying to gauge whether this is the primary cause of the severe instability or just one contributing factor, especially given that at max reasoning effort, a single turn's reasoning can balloon to thousands of tokens.

This is a nuanced position. The assistant recognizes that at reasoning_effort=max, a single turn's chain-of-thought can be thousands of tokens long. If this reasoning is being fed back into the prompt every turn, it doesn't just create OOD inputs—it also consumes massive amounts of context window, potentially crowding out the actual conversation history and tool results that the model needs to track.

The Bash Command: Tracing the Code

The message includes a bash command that executes on the remote server to investigate the deployment's git history and locate the encoding files:

ssh -o StrictHostKeyChecking=no root@10.1.230.171 '
cd /root/sglang-dsv4
echo "###### deployed HEAD + our commits ######"
git log --oneline -8
echo
echo "###### files matching encoding/dsv4 ######"
find python -path "*entrypoints/openai/encoding*" -o -name "*.py" -path "*dsv4*" 2>/dev/null | grep -iE "encoding|dsv4" | head
echo
echo "###### git diff stat vs upstream base (our patches) ######"
git diff --stat HEAD~6 2>/dev/null | tail -20
'

The results reveal the deployment's patch stack:

7e4703d98 sm120/dsv4: honor SGLANG_DEFAULT_THINKING default in chat encoding
598928d75 sm120: capture-safe Triton DSA indexer (O(actual seq), not O(context))
eb54448ab sm120 DSv4 decode optimization checkpoint (MMA split-K decode + bf16 GEMMs)
7cead0fb8 Add JonnyKong to CI_PERMISSIONS.json (#28550)
f5b041622 [AMD] Fix deepseek-v4 mtp accept length issue (#28520)
8aaca72c2 [FIX]Fix Step3-VL multi-image embedding and local patch splitting (#24970)
873196f7...

This is a critical piece of information. The top three commits are the team's custom patches:

  1. 7e4703d98: A patch to serving_chat.py that honors SGLANG_DEFAULT_THINKING in the chat encoding—this is directly relevant to how reasoning content flows through the system.
  2. 598928d75: The Triton DSA indexer, a custom CUDA kernel for attention that uses page-level early-exit.
  3. eb54448ab: The MMA split-K decode kernel and bf16 GEMM optimizations for the MoE (Mixture of Experts) layers. The assistant is specifically looking for whether the encoding behavior is from a custom patch or stock SGLang. The commit 7e4703d98 ("honor SGLANG_DEFAULT_THINKING default in chat encoding") is the most relevant—it modifies how thinking mode is handled in the encoding pipeline. But the core encoding_dsv4.py file itself might be stock SGLang's port of the reference implementation, not a custom patch.

The Thinking Process: A Model of Diagnostic Rigor

The reasoning section of message 12838 reveals a sophisticated diagnostic process. Let me trace through the assistant's thinking step by step:

Step 1: Confirm the mechanism. The assistant has already seen the evidence in message 12836—the raw message array shows reasoning_content as a separate key on the assistant message, and the templated prompt shows <think> tags that weren't in the content field. The mechanism is confirmed.

Step 2: Assess the impact. The assistant considers whether this is the primary cause or just a contributing factor. The key variable is reasoning_effort=max, which produces thousands of reasoning tokens per turn. At that scale, the leakage compounds rapidly.

Step 3: Determine ownership. The assistant needs to know whether this behavior is from stock SGLang or a custom patch. If it's stock, the fix might be different (upstream bug report vs. local patch). If it's custom, the team owns the fix directly.

Step 4: Locate the code. The bash command searches for encoding files and reviews the git history to understand which patches touch the encoding pipeline.

Step 5: Plan the next step. The assistant implicitly plans to read the actual encoding code to understand exactly where reasoning_content gets injected into the prompt, and whether the drop_thinking parameter (which controls this behavior) is being set correctly.

This is a textbook example of structured debugging: confirm the mechanism, assess impact, determine ownership, locate the code, and plan intervention.

Assumptions and Potential Mistakes

The message contains several assumptions that deserve scrutiny:

Assumption 1: DeepSeek models don't expect prior-turn reasoning in the prompt. This is stated as fact, but the assistant hasn't yet verified it against the model card or reference encoding. The user's next message ("Check model card for think strip assumptions") suggests this assumption may be incomplete or incorrect. As we see in subsequent messages (12842-12843), the reference encoding actually keeps reasoning when tools are present—it's spec-compliant behavior, not a bug.

Assumption 2: Stripping reasoning from history is the correct fix. The assistant proposes stripping reasoning_content from historical assistant messages. But if the model's training distribution actually includes prior reasoning in tool-calling scenarios (as the reference encoding suggests), this fix could itself create an OOD condition—just in the opposite direction.

Assumption 3: The encoding behavior is uniform across all requests. The assistant assumes that reasoning_content is always re-emitted as <think> tags. But the behavior might depend on parameters like drop_thinking, reasoning_effort, or the presence of tools in the request.

Assumption 4: The instability is caused by OOD inputs rather than numerical errors. The assistant is leaning toward the encoding hypothesis and away from the kernel-level hypothesis. But as the chunk summary notes, the MHC bf16 GEMM patch (which casts fp32 mixing weights to bf16 unconditionally at every layer) could also be introducing numerical drift that compounds over long contexts. The assistant hasn't ruled this out.

Input Knowledge Required

To fully understand message 12838, the reader needs knowledge of:

  1. The OpenAI chat completions API format, specifically the reasoning_content field that some models use to separate chain-of-thought from visible output.
  2. The DeepSeek chat template convention, where reasoning is enclosed in <think>...</think> tags and is expected to be stripped from prior turns in multi-turn conversations.
  3. The SGLang inference server architecture, particularly its encoding pipeline that converts OpenAI-format messages into the model's native prompt format.
  4. The opencode agent harness, which stores assistant responses including reasoning content and sends them back in subsequent requests.
  5. The concept of out-of-distribution (OOD) inputs in LLM serving—inputs that differ from the model's training distribution and can cause unpredictable behavior.
  6. The git history and patch stack of the deployment, including the custom MMA split-K, Triton indexer, and thinking-mode patches.

Output Knowledge Created

Message 12838 produces several concrete outputs:

  1. A confirmed bug mechanism: Prior-turn reasoning content is being re-emitted into the conversation history as <think> tags, creating OOD inputs for the model.
  2. A proposed fix: Strip reasoning_content from historical assistant messages when building the prompt.
  3. A code location: The encoding pipeline in python/sglang/srt/entrypoints/openai/encoding_dsv4.py is where the rendering happens.
  4. A patch inventory: The three custom patches (thinking-mode, Triton indexer, MMA split-K) are identified and their relevance to the bug is assessed.
  5. A risk assessment: The assistant identifies that at reasoning_effort=max, the leakage compounds rapidly because each turn produces thousands of reasoning tokens.

The Broader Significance

Message 12838 represents a classic debugging pivot: from blaming external factors (temperature, kernel errors) to identifying a specific, reproducible mechanism in the prompt construction pipeline. The assistant correctly identifies that the encoding layer is transforming the input in a way that violates the model's training distribution.

But the story doesn't end here. The user's next message—"Check model card for think strip assumptions"—is a crucial intervention. It tells the assistant to verify its core assumption against the official specification. And indeed, when the assistant checks the reference encoding (messages 12839-12843), it discovers that the behavior is actually spec-compliant: when tools are present, drop_thinking is automatically disabled, and prior reasoning is supposed to be retained. The bug, if there is one, lies elsewhere—perhaps in how the SGLang port diverges from the reference encoding, or in how the interrupted generation (the tic-tac-toe response was cut mid-token) interacts with the encoding logic.

This makes message 12838 a fascinating case study in the dangers of assuming you understand the model's training distribution. The assistant's reasoning was logical and internally consistent, but it was based on an assumption that turned out to be incorrect. The model was designed to handle prior reasoning in tool-calling scenarios. The real bug was something subtler—perhaps a divergence between the reference encoding implementation and the SGLang port, or an interaction between the interrupted generation and the encoding logic.

Conclusion

Message 12838 captures a pivotal moment in a complex debugging session: the discovery of a plausible mechanism for multi-turn context loss, followed by the disciplined step of tracing the mechanism to its code location. The assistant demonstrates strong diagnostic skills—connecting evidence from request logs, understanding the API format, tracing through the encoding pipeline, and proposing a concrete fix.

Yet the message also illustrates the dangers of premature certainty in debugging. The assistant's core assumption—that prior reasoning in the prompt is always OOD—would be overturned in the very next round when the user directs it to check the model card. The real lesson is that even the most logical chain of reasoning needs to be validated against the ground truth of the model's specification.

In the end, the debugging session would continue for many more rounds, eventually leading to a systematic audit of every performance patch and the creation of diagnostic tools to isolate the real cause. But message 12838 remains the moment when the assistant first connected the dots between the reasoning_content field in the API and the <think> tags in the templated prompt—a connection that would prove essential to understanding the full picture, even if the initial interpretation of that connection was incomplete.