Systematic Reverse Engineering: Deconstructing the Kimi-K2 EAGLE-3 Speculative Decoding Model

Introduction

In the rapidly evolving landscape of large language model inference, speculative decoding has emerged as one of the most impactful techniques for accelerating generation without sacrificing quality. Among the various approaches, EAGLE-3 represents a sophisticated evolution that fuses multi-layer hidden states from a target model to drive a lightweight draft head. But what does such a model actually look like under the hood? How does one go about reverse-engineering a published draft model from its HuggingFace repository, extracting not just its architecture but also its training provenance and deployment configuration?

This article examines a coding session that provides a masterclass in precisely this kind of analysis. Over the course of eight messages, an AI assistant systematically investigates the AQ-MedAI/Kimi-K2-Instruct-eagle3 model — an EAGLE-3 speculative decoding draft model designed to accelerate inference for the Kimi-K2 large language model. The investigation spans initial reconnaissance, deep architectural analysis, training infrastructure discovery, and deployment configuration synthesis. By the end, the assistant produces a comprehensive specification that any engineer could use to understand, deploy, or replicate the model.

The Research Mission: From User Request to Systematic Investigation

The session begins with a user request ([msg 0]) that is remarkably precise in its scope. The user asks the assistant to examine the AQ-MedAI/Kimi-K2-Instruct-eagle3 model on HuggingFace and return: the complete config.json contents, model file sizes, all metadata about training, the exact architecture specification, and how the model integrates with SGLang. This is not idle curiosity — the broader context reveals that the user is working on deploying speculative decoding for Kimi-K2.5 and needs to understand a reference implementation before building their own training pipeline.

The user's request encodes several implicit assumptions. First, that the HuggingFace repository contains the information needed — a reasonable assumption for a published model, though not guaranteed. Second, that the EAGLE-3 model for Kimi-K2 is structurally similar enough to what the user needs for Kimi-K2.5 that studying it will be productive. Third, that the model integrates with SGLang in a documented way, with specific flags that can be identified from the repository. These assumptions shape the entire investigation that follows.

Phase 1: Broad Reconnaissance — Gathering the Low-Hanging Fruit

The assistant's first response ([msg 1]) demonstrates a textbook approach to systematic investigation. Rather than making sequential fetches, the assistant dispatches four parallel web fetches targeting: the model's main HuggingFace page, the config.json blob view, the repository file tree, and the AQ-MedAI organization page. This parallel decomposition strategy reflects an understanding that these sources are independent — none depends on the results of another — and that batching them minimizes the number of rounds needed.

The choice of targets reveals a multi-dimensional research strategy. The model page yields the README/model card with training descriptions. The config.json provides the architectural blueprint. The file tree reveals what artifacts exist (and importantly, what doesn't exist — like a training_state.pt file). The organization page sets up the ability to compare against other EAGLE-3 models from the same publisher, a comparative analysis that becomes a major theme later.

However, this first round also reveals some suboptimal choices. The assistant fetches the blob URL for config.json (.../blob/main/config.json) rather than the raw URL (.../raw/main/config.json), which returns the JSON wrapped in HuggingFace's HTML UI. The assistant also doesn't fetch the HuggingFace API endpoint initially, which would have returned structured JSON with file listings, download counts, and metadata in a single call. These are minor but instructive mistakes that the assistant corrects in subsequent rounds.

Phase 2: Deepening the Investigation — Comparative Analysis and Raw Data

In the second round ([msg 2]), the assistant transitions from broad survey to targeted investigation. The message reads: "Good, I have the main page, config, and file listing. Now let me check the training state file and the raw config, and also look at other relevant EAGLE3 models they've published for reference." This explicit planning reveals a methodical workflow: assess what you have, identify what's missing, and fetch it.

The assistant dispatches four more parallel fetches. The most technically astute choice is the HuggingFace API endpoint (https://huggingface.co/api/models/AQ-MedAI/Kimi-K2-Instruct-eagle3), which returns structured JSON containing the model ID, download count (2,142), likes (11), license (MIT), architecture tags (LlamaForCausalLMEagle3, llama), and crucially, the siblings array — a complete listing of every file in the repository with its size. This single API call replaces what would otherwise require multiple page fetches and manual parsing.

The assistant also fetches the config.json of Qwen3-VL-235B-A22B-Instruct-eagle3, another EAGLE-3 model from the same publisher. This comparative move is strategic: by examining a different base architecture's EAGLE-3 config, the assistant can identify which parts of the configuration are standard EAGLE-3 conventions versus model-specific choices.

Phase 3: The Raw Config and Training Dataset — Filling Critical Gaps

By the third round ([msg 3]), the assistant has accumulated enough context to know exactly what it still needs. It fetches the training dataset page, a second comparative model config (Qwen2.5-VL-72B-Instruct-eagle3), and — critically — the raw config.json via the /raw/main/config.json endpoint. This last choice is a methodological correction from the first round: the raw endpoint returns pure, unformatted JSON without HTML wrappers, ensuring the assistant gets the complete, untruncated configuration.

The raw config reveals the architectural crown jewels. The architecture is LlamaForCausalLMEagle3 — a custom class registered in SGLang's inference engine, not a standard HuggingFace architecture. The eagle_config block contains eagle_aux_hidden_state_layer_ids: [2, 30, 58] and use_aux_hidden_state: true, which are the signature EAGLE-3 extensions. The draft_vocab_size: 32000 field is another non-standard parameter that signals the model uses a reduced vocabulary for its output projection — a technique associated with FR-Spec (Frequency-Reduced Speculation) that dramatically reduces the draft head's computational cost.

The config reveals that the draft model is a single-layer transformer with hidden_size=7168, intermediate_size=18432, num_attention_heads=64, and num_key_value_heads=64 — full multi-head attention rather than grouped-query attention. This makes the Kimi-K2 draft model notably larger (1.19B parameters) than comparable EAGLE-3 models for Qwen architectures, which use GQA with only 4 KV heads.

Phase 4: Weight Structure and SGLang Integration — Understanding How It Works

In the fourth round ([msg 4]), the assistant pivots from configuration to implementation. It attempts to fetch the safetensors metadata (to understand the exact weight structure) and searches for SGLang's EAGLE-3 model loading documentation. The search returns the SGLang speculative decoding documentation page, which describes EAGLE-2/EAGLE-3, MTP, classic draft-model decoding, and NGRAM-based variants.

This phase is critical because the config.json tells us what the model is, but not how it is loaded or run. The SGLang documentation provides the framework context: the --speculative-algorithm EAGLE3 flag selects the algorithm, --speculative-draft-model-path points to the draft model, and --speculative-num-steps, --speculative-eagle-topk, and --speculative-num-draft-tokens control the drafting behavior.

Phase 5: SpecForge — Tracing the Training Infrastructure

The investigation takes a strategic pivot in messages 5 and 6. The assistant recognizes that static analysis of the HuggingFace repository has reached its limits — the training hyperparameters, dataset composition, and inference integration details are simply not exposed through HuggingFace metadata. The assistant turns to SpecForge, the official training and deployment framework for EAGLE models within the SGLang project.

In message 5, the assistant searches for "SpecForge eagle3 config Kimi-K2 draft model architecture eagle_aux_hidden_state_layer_ids training." The search returns the SpecForge pull requests page, revealing PR #381 ("Support Qwen3,Qwen3-Next,Kimi-K2,Deepseek models template") and PR #378 ("feat: add training support for DeepSeek-V3 EAGLE-3 speculative decoding"). The presence of PR #381 — explicitly mentioning Kimi-K2 — confirms that SpecForge has recently added support for this model, pointing the assistant toward the actual configuration and code changes needed.

In message 6, the assistant dives deeper by fetching two specific SpecForge PRs: PR #108 ("Added Eagle training support for Kimi-K2") and PR #381 (the template support PR). These PRs represent the bridge between static model inspection and understanding the training and deployment infrastructure. PR #108 is the foundational PR that enabled training the model; PR #381 addresses inference-side integration. Together, they span the full lifecycle.

Phase 6: The SpecForge Config — The Missing Piece

Message 7 represents the final data-gathering step before synthesis. The assistant guesses the URL for the SpecForge configuration file: https://raw.githubusercontent.com/sgl-project/SpecForge/main/configs/kimi-k2-eagle3.json. This educated guess is based on naming conventions seen in similar projects and the PR titles. The assistant also fetches the configs/ directory listing as a fallback — if the guessed filename is wrong, the directory listing will reveal the actual naming convention.

This message demonstrates a sophisticated understanding of risk management. The assistant is not certain the config file exists at the guessed URL, so it fetches the directory listing alongside the specific file. This is a robust investigative pattern: always gather context alongside specific targets.

Phase 7: Synthesis — The Comprehensive Analysis

Message 8 is the culmination of the entire investigation. The assistant synthesizes information gathered across seven prior research messages — spanning web fetches, HuggingFace API queries, GitHub PR examinations, and code context searches — to produce a comprehensive specification of the AQ-MedAI/Kimi-K2-Instruct-eagle3 model.

The analysis is structured into ten clearly delineated sections. It begins with the complete config.json reproduced verbatim — the foundational artifact from which all subsequent analysis flows. It then presents the repository file listing (just four files totaling 2.38 GB), the parameter breakdown from safetensors metadata (1,191,144,704 parameters total), and the exact architecture specification in a structured table.

The heart of the analysis is the architectural interpretation. The assistant explains that the draft head fuses hidden states from three layers of the Kimi-K2 target model — layer 2 (early features), layer 30 (mid-level features), and layer 58 (high-level features) — which is the signature EAGLE-3 innovation over EAGLE-2's single-layer approach. The choice of layers is not arbitrary: they span the model's depth roughly evenly, suggesting a design principle of capturing multi-scale representations.

The assistant also provides the practical payoff: the exact SGLang launch command with all necessary flags. The command includes --speculative-algorithm EAGLE3, --speculative-draft-model-path, --speculative-num-steps 3, --speculative-eagle-topk 1, and --speculative-num-draft-tokens 4. A flags table explains what each parameter does and why it is set to its value.

The comparative analysis with five other AQ-MedAI EAGLE-3 models reveals important patterns. All models use the LlamaForCausalLMEagle3 architecture, suggesting a standardized approach. The Kimi-K2 draft model is notably larger (1.19B) because Kimi-K2 uses full MHA rather than GQA. The Kimi-K2 model also lacks target_model_type and step_num fields that newer configs include, suggesting it was an earlier release.

Conclusion

This coding session demonstrates a systematic approach to reverse-engineering a complex ML model. The assistant progresses through distinct phases — broad reconnaissance, targeted data gathering, comparative analysis, infrastructure tracing, and synthesis — each building on the previous. The parallel fetch strategy, the educated guesses about file locations, the use of structured API endpoints over rendered HTML, and the cross-referencing of multiple information sources all contribute to a comprehensive understanding.

The final output is more than a data dump — it is a curated synthesis that transforms raw observations into structured, actionable knowledge. Any engineer could reconstruct the draft model, deploy it with SGLang, or use the analysis as a template for investigating similar models. The session stands as a model of methodical technical analysis in the AI engineering domain.