Probing the API: How a Raw curl Request Uncovered the Reasoning Format for EAGLE-3 Training Data

Introduction

In the middle of a complex EAGLE-3 speculative decoding training pipeline for the 1-trillion-parameter Kimi-K2.5 model, a seemingly simple data quality issue triggered a precise diagnostic maneuver. The user had noticed that the synthetic training data being generated by the inference script contained empty reasoning fields — the model's rich internal chain-of-thought was being discarded. The assistant's response, message 2921 in the conversation, was a single bash command: a raw curl request to the vLLM API endpoint, bypassing the OpenAI client library entirely. This message, though brief, represents a critical turning point in the debugging process and reveals much about the assistant's methodology, assumptions, and the knowledge boundaries at play.

The Context: Building EAGLE-3 Training Data

The broader project was ambitious. The team had deployed the Kimi-K2.5 INT4 quantized model across 8 NVIDIA RTX PRO 6000 Blackwell GPUs and was working to train an EAGLE-3 speculative decoding drafter — a lightweight draft model that predicts the next several tokens in parallel, allowing the main model to verify them in a single forward pass. The promise of EAGLE-3 was significant throughput gains without sacrificing output quality, but it required high-quality training data that captured the actual reasoning behavior of the target model.

The synthetic data generation pipeline worked as follows: the script 01b_generate_synthetic.py fed questions from the mlabonne/open-perfectblend dataset to the vLLM inference server one by one (at concurrency 128), captured the model's responses, and saved them for later hidden state extraction and training. The critical requirement was that each response needed to preserve the model's reasoning — the internal chain-of-thought that Kimi-K2.5 generates before producing its final answer. In the Kimi-K2.5 architecture, reasoning is demarcated by special tokens: <think> (token ID 163606) and </think> (token ID 163607), which wrap the model's internal deliberation before the final response token.

The Problem: Empty Reasoning

When the user inspected the output from the first run (which had been killed after accumulating 388 samples and 222 timeout errors out of ~2700 attempted), they found that every sample had "reasoning": "" — an empty string. The model's reasoning was simply not being captured. The content field contained the final answer (e.g., "41 claims" for a math word problem), but the chain-of-thought that led to that answer was missing. This was a fatal flaw for EAGLE-3 training: the drafter needs to learn the full generation process, including the reasoning tokens, not just the final answer.

The user raised two issues simultaneously: cap the run at 10K samples (rather than 25K), and fix the reasoning capture. They also noted that when reassembling the full token sequence for hidden state extraction, the correct <think> and </think> tokens needed to be inserted around the reasoning content.

The Assistant's Initial Hypothesis

Before message 2921, the assistant had already formulated a hypothesis in message 2919. It reasoned that the vLLM kimi_k2 reasoning parser likely puts the reasoning content into a field called reasoning_content (following OpenAI's API convention), but that the OpenAI Python client might not expose this field correctly. The assistant also noted that the completions were suspiciously short — 280 tokens for a math word problem — suggesting the model was answering without reasoning at all, possibly because the prompt didn't instruct it to think.

This hypothesis contained two assumptions:

  1. That the field name was reasoning_content (following OpenAI's extended API convention for reasoning models)
  2. That the model might not be reasoning because the prompt lacked an explicit thinking instruction Both assumptions were reasonable but ultimately incorrect on the first point and incomplete on the second.

The Diagnostic: Message 2921

Message 2921 is the assistant's response to this impasse. Rather than continuing to speculate or making another round of edits based on assumptions, the assistant chose to probe the API directly. The message contains a single tool call — a bash command that executes a curl request to the vLLM chat completions endpoint:

ssh root@10.1.230.174 'curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"/shared/kimi-k2.5-int4\",
    \"messages\": [{\"role\": \"user\", \"content\": \"What is 2+2?\"}],
    \"max_tokens\": 512,
    \"temperature\": 0.6
  }" | python3 -m json.tool'

This is a textbook debugging move. The assistant deliberately bypasses the OpenAI Python client library — which could be doing any number of transformations, field mappings, or omissions — and talks directly to the HTTP API. The python3 -m json.tool at the end ensures the response is pretty-printed for human inspection.

The choice of prompt is also telling: "What is 2+2?" is the simplest possible question. It minimizes variables. If the model produces reasoning for this trivial question, then reasoning is working at the API level and the problem is in the client library. If it doesn't, the issue is deeper (perhaps the model configuration or prompt format).

What the Response Revealed

The API response, partially visible in the message, shows the raw JSON structure:

{
    "id": "chatcmpl-9784cf4f6f38a9d1",
    "object": "chat.completion",
    "created": 1771743186,
    "model": "/shared/kimi-k2.5-int4",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": " 2 + 2 = **4**",
                "refusal": null,
                "annotations": null,
                "audio": null,
                "function_call": null,
                "tool_calls": [],
                "reasoning":...

The critical detail is that the field is called reasoning, not reasoning_content. The message is truncated in the conversation display, but the field name is clearly visible. This single observation invalidated the assistant's first assumption: the script was checking msg.reasoning_content (or response.choices[0].message.reasoning_content in OpenAI client terms), but the actual field in the API response is reasoning.

The assistant's subsequent message (2922) confirmed this definitively by running a Python test with the OpenAI client, printing all fields of the response object. It found that msg.reasoning contained the full reasoning text while msg.reasoning_content was absent. The reasoning for "What is 2+2?" was a thoughtful paragraph about providing a clear, direct answer — proving that the model was reasoning even for trivial questions, and that the issue was purely a field name mismatch in the Python script.

Knowledge Boundaries and Assumptions

This message illuminates several important knowledge boundaries:

Input knowledge required to understand this message:

The Thinking Process

The reasoning visible in this message is concise but reveals a clear methodology:

  1. Hypothesis formation: The assistant had already formed two hypotheses in the previous message (message 2919) about why reasoning was empty.
  2. Direct verification: Rather than making another code change based on assumptions, the assistant chose to verify the API response format directly.
  3. Minimal test case: The prompt "What is 2+2?" is deliberately trivial — if reasoning appears here, it confirms the API is working correctly and the issue is in the client library.
  4. Raw access: Using curl instead of the Python client removes all abstraction layers, giving the assistant direct access to the server's actual response.
  5. Pretty-printing: The python3 -m json.tool pipeline ensures the JSON is readable, making it easy to spot the field name.

Impact and Resolution

The impact of this single diagnostic message was immediate and significant. In the following messages (2922-2924), the assistant:

  1. Confirmed that msg.reasoning (not msg.reasoning_content) contains the reasoning text
  2. Discovered that <think> maps to token ID 163606 and </think> maps to token ID 163607 in the Kimi-K2.5 vocabulary
  3. Fixed the Python script to use the correct field name
  4. Added logic to wrap reasoning content with the correct special tokens during reconstruction
  5. Capped the run at 10K samples as requested The fix was then deployed, and the synthetic data generation restarted with correct reasoning capture. The 388 samples from the previous run were lost (the old script only wrote output at the end, not streaming), but the new script streamed results to disk incrementally, preventing future data loss.

Broader Lessons

This message exemplifies a fundamental debugging principle: when a library abstraction produces unexpected results, bypass the abstraction and inspect the raw data. The OpenAI Python client is a convenient wrapper, but it can silently drop fields, rename attributes, or fail to parse extensions to the API schema. By going directly to the HTTP API with curl, the assistant eliminated an entire class of potential bugs in a single command.

The message also demonstrates the value of minimal test cases. A simple "What is 2+2?" prompt isolates the reasoning mechanism from confounding factors like prompt complexity, context length, or multi-turn conversation structure. When debugging complex systems, reducing the problem to its simplest reproducible form is often the fastest path to resolution.

Finally, this message highlights the importance of understanding the full data pipeline in machine learning systems. The EAGLE-3 training pipeline had multiple stages: inference → JSONL storage → hidden state extraction → training. A bug in the first stage (incorrect field name) would corrupt all downstream stages. Catching it early, before investing hours in generating 25K samples, saved enormous wasted computation.