The Reasoning Parser Discovery: A Pivot from Client-Side Parsing to Server-Side Configuration
Introduction
In the course of building a large-scale training dataset for the Kimi-K2.5 language model, a subtle but critical bug emerged: the reasoning content — the model's internal chain-of-thought — was not being captured in the generated responses. The user spotted the problem when inspecting sample outputs from the inference pipeline: the reasoning field was empty, and the thinking content was instead embedded directly in message.content. What followed was a classic debugging journey that led from a client-side parsing fix to the discovery of a proper server-side configuration option. The message at [msg 3770] represents the pivotal moment when the assistant confirmed the root cause and began investigating the correct solution.
Context: The Training Data Pipeline
The broader project involved generating synthetic training data for the Kimi-K2.5 model using SGLang, a high-performance inference engine. The pipeline, orchestrated by run_inference.py, took prepared prompts from various datasets (B1_glaive, B2_opencodeinstruct, etc.), sent them to an SGLang server running on an 8-GPU machine, and saved the responses as tokenized training data. The assistant had already run this pipeline for the B1_glaive dataset, producing 2,900 completed responses out of 10,000 prompts.
When the user inspected sample outputs ([msg 3746]), they noticed that the reasoning field was empty for both examples shown. The first sample — a code generation task — had an empty reasoning string despite the model clearly having performed reasoning (the response content began with "The user wants a function..."). The second sample — a restaurant search with function calling — similarly showed reasoning: "". This was a regression: the user explicitly noted "this was also a bug previously fixed in 10k version."
Initial Investigation: Client-Side Parsing
The assistant's first response ([msg 3747]) was to examine run_inference.py. The code at line 59 used reasoning = getattr(msg, "reasoning", None) or "", which relied on the OpenAI-compatible chat completions API exposing a reasoning attribute on the message object. The assistant hypothesized that SGLang's response format differed from OpenAI's.
A series of SSH commands confirmed the problem (<msg id=3748-3750>). The raw response files showed reasoning was always empty, but the content field contained the thinking text — including the response tag that separates reasoning from the final answer. The content started directly with the thinking text (no opening thinking tag), had response somewhere in the middle, and then the actual response.
The assistant formulated a client-side fix ([msg 3761]): parse the content field by splitting on response, treat everything before as reasoning and everything after as content, and prepend thinking since the model's output begins with thinking but SGLang strips the opening tag.
The User's Insight
At [msg 3763], the user interjected with a crucial observation: "Pretty sure we're running sglang with wrong reasoning parser(?)" This was a key insight — rather than hacking around the problem on the client side, perhaps SGLang itself had a server-side configuration option to properly split reasoning content.
The assistant immediately pivoted ([msg 3764]) and began searching SGLang's documentation. Running sglang.launch_server --help revealed the existence of --reasoning-parser with kimi_k2 as one of the available options ([msg 3769]). Checking the running server's process list confirmed the suspicion: the server was started with no --reasoning-parser flag at all.
The Subject Message: Confirmation and Deeper Investigation
The subject message ([msg 3770]) marks the moment of confirmation and the beginning of deeper investigation:
Confirmed — no--reasoning-parserflag. We need to restart with--reasoning-parser kimi_k2. Let me also check what that parser does to confirm it's the right one:
>
[bash] ssh root@10.1.230.174 'grep -r "kimi_k2" /root/sglang/python/sglang/srt/reasoning_parser.py 2>/dev/null || find /root/sglang/python/sglang -name ".py" -exec grep -l "kimi_k2.pars\|reasoning_parser.*kimi" {} \;' 2>&1 /root/sglang/python/sglang/srt/entrypoints/openai/serving_chat.py
This message is deceptively simple but represents a critical decision point. The assistant has identified the root cause — a missing server flag — and is now taking the prudent step of verifying that kimi_k2 is indeed the correct parser before restarting the server. The grep command searches for references to kimi_k2 in the reasoning parser module and related files, finding a hit in serving_chat.py (though the actual parser class is in a different file).
Assumptions and Knowledge
Several assumptions underpin this message. First, the assistant assumes that kimi_k2 is the correct parser for the Kimi-K2.5 model — a reasonable assumption given the naming, but one that requires verification. Second, the assistant assumes that the parser will properly split thinking/ response content into reasoning_content and content fields in the API response. Third, there's an implicit assumption that restarting the server with this flag will not introduce new issues or degrade performance.
The input knowledge required to understand this message includes: familiarity with SGLang's server architecture and command-line options, understanding of how reasoning models structure their output with thinking/ response tags, knowledge of the OpenAI chat completions API format (particularly reasoning_content), and awareness of the training data pipeline's dependency on properly separated reasoning content.
The Thinking Process Revealed
The reasoning visible in this message shows a methodical approach to debugging. Rather than immediately restarting the server with the new flag, the assistant pauses to verify. The grep command is structured with a fallback: first try searching specifically in reasoning_parser.py, and if that fails (no matches), search more broadly across all Python files. This reveals an understanding that the parser registration might not be in the parser module itself but in the entrypoint code that wires everything together.
The assistant is also implicitly making a cost-benefit calculation. Restarting the SGLang server is expensive — it takes significant time to load the 8-shard model across GPUs, and the inference pipeline was already paused. Getting the parser wrong would mean another restart cycle. Hence the verification step.
What Followed
The subsequent messages (<msg id=3771-3775>) traced the parser implementation. The assistant found reasoning_parser.py at a different path, discovered that kimi_k2 mapped to Qwen3Detector (which uses standard thinking/ response tags), while kimi mapped to KimiDetector (which uses Unicode variants ◁think▷/◁/think▷). This confirmed that kimi_k2 was correct for Kimi-K2.5, which uses the standard tag format.
The assistant then proceeded to kill the server, restart with --reasoning-parser kimi_k2, clear the corrupted B1_glaive data, and resume inference — a sequence that consumed the next several messages.
Output Knowledge Created
This message created several important outputs. First, it confirmed the root cause of the reasoning capture bug: the SGLang server was missing the --reasoning-parser flag. Second, it identified the correct flag value (kimi_k2) for the Kimi-K2.5 model. Third, it located the relevant source code files for further investigation. Fourth, it established a verification methodology — checking the server process list against the available command-line options — that could be applied to similar configuration issues in the future.
Mistakes and Corrective Insights
The initial approach of fixing the problem on the client side (parsing response from content) was not necessarily wrong — it would have worked as a workaround. But the user's insight about the reasoning parser was superior because it fixed the problem at the source. The server-side solution ensures that all downstream consumers of the API get properly structured responses, not just run_inference.py. This is a classic lesson in debugging: when a system component produces malformed output, check whether the component has configuration options to produce correct output before building compensating logic elsewhere.
Conclusion
The message at [msg 3770] exemplifies a crucial debugging pattern: the transition from building a workaround to finding the correct configuration. The assistant had formulated a client-side parsing fix, but the user's domain knowledge about SGLang's reasoning parser capability led to a cleaner solution. The message itself is the hinge point — the moment of confirmation that triggers a cascade of verification, server restart, and pipeline recovery. It demonstrates that effective debugging requires not just technical skill but also the willingness to pause and verify before acting, especially when the action (restarting an 8-GPU inference server) carries significant time cost.