Probing the SGLang API: How a Single Curl Command Uncovered a Reasoning Capture Bug

The Message

ssh root@10.1.230.174 'curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "{\"model\": \"/shared/kimi-k2.5-int4\", \
       \"messages\": [{\"role\": \"user\", \"content\": \"What is 2+2?\"}], \
       \"max_tokens\": 200, \"temperature\": 0}" \
  2>/dev/null | python3 -m json.tool'

This single bash command, issued by the assistant in [msg 3753], represents a pivotal diagnostic moment in a large-scale machine learning pipeline. On the surface, it is a simple curl request to an SGLang server asking "What is 2+2?" — a trivial question for any language model. But the purpose was anything but trivial: it was a surgical probe designed to reveal why an entire 88,000-sample training data generation pipeline was silently producing corrupted data.

The Context: A Pipeline Running on Faulty Assumptions

To understand why this message matters, we must step back and examine the broader project. The team was building an EAGLE-3 speculative decoding drafter for the Kimi-K2.5 model — a 1-trillion-parameter reasoning model running across 8 NVIDIA RTX PRO 6000 Blackwell GPUs. The EAGLE-3 drafter is a lightweight auxiliary model that predicts the base model's future tokens, enabling faster inference. Training such a drafter requires high-quality data: for each prompt, the pipeline must capture both the model's reasoning (the internal chain-of-thought, enclosed in thinking ... </think> tags) and its final response.

The inference pipeline, orchestrated by run_inference.py, was churning through 88,000 prompts across ten datasets, sending them to an SGLang server via OpenAI-compatible chat completions API calls. The script extracted the response using reasoning = getattr(msg, "reasoning", None) or "" — assuming the OpenAI client would expose a reasoning attribute on the message object. This assumption had worked in a previous 10,000-sample pipeline, but something had changed.

The user spotted the problem in [msg 3746], where two sample outputs showed reasoning: "" — the reasoning field was completely empty. The thinking content was nowhere to be found. The user's succinct diagnosis: "Seems again we're not capturing reasoning correctly (this was also a bug previously fixed in 10k version)."

The Investigation Before the Probe

The assistant's first instinct was to inspect the raw data files on disk. In [msg 3748], it checked the raw_responses.jsonl file and confirmed that reasoning was indeed an empty string in every record. Worse, the content field appeared to contain the thinking text — the response started with "The user is asking..." rather than with the actual answer, suggesting the thinking chain was embedded in content rather than separated into a dedicated reasoning field.

Further inspection in <msg id=3749-3750> confirmed the pattern: the &lt;/think&gt; tag was present inside content at position ~314, but there was no opening &lt;think&gt; tag. The model's output began directly with the reasoning text, and SGLang was not wrapping it in a &lt;think&gt; tag or separating it into reasoning_content.

This was the moment of uncertainty. The assistant had two competing hypotheses:

  1. The OpenAI client library was stripping the reasoning field: Perhaps the raw HTTP response from SGLang did contain reasoning_content, but the OpenAI Python client was discarding it or not exposing it properly.
  2. SGLang itself was not providing reasoning separately: Perhaps the server was configured without --reasoning-parser or some equivalent flag, causing it to dump everything into content and leave reasoning_content as null. The user's command at [msg 3752] — "Make a raw requests and see the response" — cut through the ambiguity. The only way to resolve the question was to bypass the OpenAI client entirely and inspect the raw HTTP response from the server.

The Message Itself: A Deliberate Diagnostic Probe

The assistant's response in [msg 3753] is a model of focused debugging. It constructs a minimal, self-contained curl command that hits the SGLang server's /v1/chat/completions endpoint with a trivial prompt ("What is 2+2?"). The choice of prompt is deliberate: it is simple enough that the model's response will be short and predictable, making the JSON output easy to parse and analyze. The max_tokens=200 and temperature=0 parameters ensure deterministic, concise output.

The command pipes the raw JSON response through python3 -m json.tool, which pretty-prints it. This is not just cosmetic — it makes the response structure immediately visible, revealing every field and nested object that SGLang returns.

The Assumptions Embedded in the Probe

Every diagnostic tool carries assumptions, and this curl command is no exception:

  1. The SGLang server is running and responsive: The command assumes the server at localhost:8000 on the remote machine is alive and serving requests. This was a safe assumption given that the inference pipeline was actively running.
  2. The OpenAI-compatible endpoint returns the same format as the chat client library: This is the very assumption being tested. The assistant implicitly assumes that the raw HTTP response will reveal fields that the OpenAI client might be hiding.
  3. The model will respond to a trivial prompt without errors: The prompt "What is 2+2?" is so simple that any failure would indicate a systemic server problem rather than a prompt-specific issue.
  4. SSH access and curl are available: The command assumes the remote machine has curl installed and that SSH authentication works without interactive input.

What the Probe Revealed: The Output Knowledge

The result of this curl command (visible in [msg 3760]) was unambiguous. The raw JSON response showed:

{
    "choices": [{
        "message": {
            "role": "assistant",
            "content": " The user is asking a simple math question...",
            "reasoning_content": null
        }
    }],
    "usage": {
        "reasoning_tokens": 0
    }
}

The critical finding was that reasoning_content was null and reasoning_tokens was 0. SGLang was not splitting the reasoning into a separate field at all. The entire response — both reasoning and final answer — was concatenated into message.content, with &lt;/think&gt; marking the boundary between the two sections. Notably, the opening &lt;think&gt; tag was absent; the model's output began directly with the reasoning text.

This output knowledge completely reframed the problem. The bug was not in the OpenAI client library or in run_inference.py's parsing logic per se — it was a server-side configuration issue. SGLang needed to be started with a --reasoning-parser flag (or equivalent) to enable the separation of reasoning content. Without it, the server treated the model's raw output as a single content blob.

The Thinking Process: From Symptom to Root Cause

The assistant's reasoning chain, visible across messages 3747-3753, demonstrates a systematic debugging methodology:

  1. Observe the symptom: Reasoning field is empty in captured data.
  2. Check the raw data: Inspect raw_responses.jsonl on disk to confirm the symptom is not a display artifact.
  3. Characterize the pattern: Look at multiple samples to see if the pattern is consistent (it is).
  4. Examine the content structure: Search for &lt;/think&gt; tags inside content to understand how the model's output is structured.
  5. Formulate hypotheses: Two competing explanations — client-side stripping vs. server-side omission.
  6. Design a decisive experiment: Issue a raw HTTP request to the server, bypassing all client libraries.
  7. Execute the probe: The curl command in [msg 3753].
  8. Interpret the result: reasoning_content: null confirms the server is the source of the problem. This is textbook debugging: start with the most direct observation, narrow the hypothesis space, and design an experiment that cleanly distinguishes between competing explanations.

The Impact: A Fork in the Pipeline

The discovery had immediate and far-reaching consequences. The entire inference pipeline had to be stopped (the user requested a pause in [msg 3754], and the assistant killed the process in <msg id=3758-3759>). The 2,900 responses already generated for the B1_glaive dataset were effectively useless because they lacked properly separated reasoning content.

More fundamentally, the team faced a choice: either reconfigure SGLang to properly emit reasoning_content, or rewrite run_inference.py to parse the &lt;/think&gt; tag from message.content. The assistant ultimately chose the latter approach (documented in [msg 3761]), implementing a parser that splits on &lt;/think&gt; and prepends &lt;think&gt; to the reasoning portion. This decision was pragmatic — it avoided the need to restart the SGLang server with different flags, which would have disrupted other users and required re-caching the model.

Conclusion

The curl command in [msg 3753] is a masterclass in diagnostic minimalism. In a single line, it cut through layers of abstraction — the OpenAI client library, the Python JSON parser, the application-level logging — to reveal the raw truth of what the server was returning. It transformed a vague symptom ("reasoning is empty") into a precise, actionable diagnosis ("SGLang is not providing reasoning_content; we must parse &lt;/think&gt; from content").

This message also illustrates a deeper truth about debugging complex ML systems: the most powerful tool is often the simplest one. Before diving into code changes, before rewriting the inference script, before reconfiguring the server — just ask the server directly what it's sending. The answer, in this case, was hiding in plain sight, waiting for a single curl command to reveal it.