The Draft Model Detective: How an AI Agent Systematically Hunted for Speculative Decoding Candidates for Kimi-K2.5

Introduction

In the rapidly evolving landscape of large language model deployment, few optimization techniques offer the latency-reduction potential of speculative decoding. The core idea is elegant: pair a massive "target" model with a much smaller, faster "draft" model that shares the same vocabulary. The draft model generates candidate tokens quickly, and the target model verifies them in a single parallel forward pass, accepting or rejecting each proposal. When the draft model's predictions align well with the target model's distribution, effective throughput can double or even triple.

But finding the right draft model is far from trivial. It requires navigating a complex landscape of tokenizer compatibility, architecture alignment, model availability, and practical deployment constraints. This article examines a complete subagent session within an opencode coding session where an AI assistant was tasked with finding candidate draft models for Kimi-K2.5 — a 1-trillion-parameter Mixture-of-Experts (MoE) model developed by Moonshot AI, built on the DeepSeek V3 architecture with Multi-head Latent Attention (MLA). Over the course of nine messages, the assistant conducted a systematic, multi-round investigation that culminated in a comprehensive analysis of seven candidate approaches organized across four tiers. The journey from initial broad searches to a structured decision framework reveals much about how AI-assisted research works in practice — and why the most critical discovery was not a model at all, but a negative finding about tokenizer incompatibility.

The Research Challenge

The user's request in the opening message (msg id=0) laid out a precise research agenda. The assistant needed to find candidate draft models for speculative decoding with Kimi-K2.5, subject to three critical constraints. First, the draft model must share the same tokenizer/vocabulary as Kimi-K2.5, which uses a vocabulary of 163,840 tokens via the Kimi/MoonShot tokenizer. This is non-negotiable in speculative decoding because the draft model's token proposals must map to valid token IDs in the target model's vocabulary — if the vocabularies differ, the verification mechanism breaks down entirely. Second, the draft model must be much smaller than the 1T-parameter target — ideally fitting on a single GPU or even a fraction of one, since it needs to run alongside the target model during inference. Third, architecture compatibility is preferred: ideally the same DeepSeek V3 / MLA architecture family but smaller, or at minimum a simple dense transformer that shares the vocabulary.

The user also asked the assistant to investigate several specific questions: whether Kimi-K2.5's tokenizer is the same as DeepSeek V3's (which would open up DeepSeek's smaller models as candidates), whether any small models exist in the Kimi/MoonShot family on HuggingFace, whether DeepSeek has small dense models with the same tokenizer, whether any community-created draft models or distilled versions exist, and whether any Eagle/Medusa-style draft heads have been trained for DeepSeek V3 or similar MoE models.

This was not a simple question with a single answer. It was a research project that required gathering information from multiple sources — HuggingFace model repositories, GitHub projects, technical blog posts, and web discussions — and synthesizing that information into actionable recommendations.## The Parallel Search Strategy

The assistant's first response (msg id=1) launched four parallel web searches designed to cover the problem space from different angles. This parallel execution is a deliberate strategic choice enabled by the opencode session architecture, where all tool calls within a single round are dispatched simultaneously and the assistant waits for all results before proceeding. By decomposing the research question into independent sub-queries, the assistant maximized information throughput in a single round.

The four searches targeted distinct dimensions of the problem. The first query — "Kimi-K2.5 tokenizer vocabulary size 163840 same as DeepSeek V3" — aimed to resolve the foundational question of tokenizer compatibility. The second — "Kimi K2 small model variants HuggingFace draft speculative decoding" — searched for existing draft models in the Kimi family. The third — "DeepSeek V3 draft model speculative decoding small dense model same tokenizer" — explored whether DeepSeek's own small models could serve as drafters. The fourth — "Eagle Medusa draft head DeepSeek V3 R1 speculative decoding" — investigated specialized draft head architectures.

The results were a mixed bag. The tokenizer query returned a HuggingFace commit page for the Kimi-K2.5 repository, which confirmed the model exists but provided no direct information about tokenizer compatibility. The small model variants query was more fruitful, surfacing jukofyork/Kimi-K2-Instruct-DRAFT-0.6B-v3.0-GGUF — a community-created 0.6B parameter draft model specifically designed for Kimi-K2-Instruct, tagged with "draft" and "speculative-decoding." The DeepSeek V3 query returned a Reddit post about speculative decoding with llama.cpp — tangentially relevant but not directly answering the question. The Eagle/Medusa query returned a Perplexity blog post about accelerating their Sonar model, which was informative but not specific to DeepSeek V3 or Kimi-K2.5.

Deepening the Investigation

Message 2 marked the transition from broad exploration to targeted investigation. The assistant acknowledged the initial findings — "Good initial results. I found a key lead: jukofyork/Kimi-K2-Instruct-DRAFT-0.6B-v3.0 and SpecBundle/EAGLE-3 models" — and launched a second wave of four parallel tool calls designed to drill deeper into the most promising leads.

The assistant fetched the full HuggingFace page for the jukofyork draft model, searched for SpecBundle and EAGLE-3 models related to DeepSeek V3 and Kimi K2, searched for the critical tokenizer comparison between DeepSeek V3 (vocab_size=129,280) and Kimi K2 (vocab_size=163,840), and searched for "transplant-vocab" — a vocabulary transfer tool also created by jukofyork. This last query was particularly clever: the assistant recognized that understanding how the draft model was created would be essential for assessing its quality and reproducibility.

Message 3 continued the deepening trend. The assistant fetched the AQ-MedAI/Kimi-K2-Instruct-eagle3 model page — an EAGLE-3 draft head specifically trained for Kimi-K2-Instruct — and the LMSYS SpecBundle blog post describing their production-grade speculative decoding model collection. It also ran searches to further probe the tokenizer difference and to check whether Moonshot AI had released any small dense models. The search for "Moonshot AI Kimi small dense model 1B 7B open source HuggingFace moonshotai" returned moonshotai/Kimi-Audio-7B, which turned out to be an audio model with a different architecture — not useful for text-based speculative decoding.

By message 4, the assistant was executing focused verification queries. It searched for the specific SpecBundle EAGLE-3 model for Kimi-K2 (lmsys SGLang-EAGLE3-Kimi-K2-Instruct) and ran targeted searches to confirm the tokenizer difference using quoted vocab_size values. The SpecBundle search returned the LMSYS collection page, confirming that production-grade draft models existed in the ecosystem. The tokenizer searches continued to surface configuration files that hinted at incompatibility, but the assistant had not yet obtained definitive primary-source evidence.## The Pivotal Verification: Tokenizer Incompatibility Confirmed

Messages 5 and 6 represent the critical turning point of the entire investigation. In message 5, the assistant fetched the raw config.json from the official Moonshot AI repository for Kimi-K2-Instruct. The configuration file revealed bos_token_id: 163584 and eos_token_id: 163585, consistent with a vocabulary size of 163,840. In message 6, the assistant fetched the config.json from DeepSeek-V3-0324, which showed bos_token_id: 0 and eos_token_id: 1 — radically different values that strongly implied a different tokenizer.

This was the moment the investigation crystallized. The assistant could now definitively state that Kimi-K2.5 and DeepSeek V3 use different tokenizers — a gap of 34,560 tokens between their vocabularies, with completely different special token IDs. This single finding had cascading implications: DeepSeek's own small models, DeepSeek V3's native Multi-Token Prediction (MTP) module, and any draft models trained on the DeepSeek vocabulary were all ruled out for direct use with Kimi-K2.5. The search space narrowed dramatically.

The assistant's methodological choice here is worth highlighting. Rather than relying on secondary sources, blog posts, or community discussions about the tokenizer, the assistant went directly to the authoritative source — the raw config.json files on HuggingFace. This is a principle that applies broadly in technical research: when the question is about a specific numerical property of a model, the most reliable source is the model's own configuration file, not someone else's summary of it.

The Final Gap-Filling and Synthesis

Message 7 marked the transition from research to synthesis. The assistant declared "Now I have all the data I need" and launched two final targeted searches: one for a Kimi-K2.5-specific draft model (which returned the same AQ-MedAI EAGLE-3 model, confirming no K2.5-specific model existed), and one for DeepSeek V3's native MTP capability (which returned a tutorial about SGLang speculative decoding for DeepSeek models). These two searches filled the remaining gaps: there was no K2.5-specific draft model available, and while DeepSeek V3 had native MTP, Kimi-K2.5 did not inherit it (as confirmed by num_nextn_predict_layers=0 in the Kimi-K2-Instruct config).

Message 8 delivered the comprehensive analysis that was the culmination of all seven preceding rounds of research. The assistant organized the findings into a four-tier taxonomy that transformed a flat list of model names into a structured decision framework.

Tier 1: Production-Ready, Specifically Built for Kimi-K2 contained two entries. The first was AQ-MedAI/Kimi-K2-Instruct-eagle3, a publicly available ~1B parameter EAGLE-3 draft head trained specifically for Kimi-K2-Instruct, part of the LMSYS SpecBundle collection, achieving up to 1.8x peak throughput with acceptance lengths of 3.1-3.5 tokens. The assistant noted that since K2.5 is built on K2-Base, this model may partially transfer, but retraining for K2.5 specifically would be recommended. The second entry was Baseten's custom EAGLE-3 speculator for Kimi-K2.5, which achieved 340+ tokens/second but was proprietary and not publicly released.

Tier 2: Community-Created Vocab-Transplanted Draft Models featured jukofyork/Kimi-K2-Instruct-DRAFT-0.6B-v3.0, a 0.65B parameter Qwen2-based dense transformer that used the transplant-vocab tool to map Qwen2.5-0.5B-Instruct's vocabulary onto the Kimi-K2 tokenizer, then fine-tuned for one epoch on ~2.3B tokens. This model worked with llama.cpp and was available in both Safetensors and GGUF formats.

Tier 3: DeepSeek V3 Draft Models (INCOMPATIBLE) served as a graveyard of otherwise promising candidates killed by the tokenizer mismatch. The assistant explicitly listed DeepSeek V3's native MTP module and jukofyork/DeepSeek-V3-DRAFT-0.6B-v3.0, both marked with clear "INCOMPATIBLE" labels and explanations of why they wouldn't work.

Tier 4: DIY / Training Your Own presented two forward-looking approaches. The recommended approach was training a new EAGLE-3 head for K2.5 via SpecForge, following the same methodology Baseten used for their production deployment. The alternative was creating a transplanted dense draft model via transplant-vocab, which would work with llama.cpp but achieve lower acceptance rates than EAGLE-3.## The Summary Table: A Decision Matrix

The summary table in message 8 condensed seven candidates into a single matrix with eight columns: model identifier, size, type, tokenizer match, framework, throughput gain, and K2.5 readiness. The "K2.5 Ready?" column was particularly nuanced, containing assessments like "Partial (trained for K2)" for the AQ-MedAI EAGLE-3 model and "Best option" for training a new EAGLE-3 head via SpecForge. These assessments reflected the assistant's understanding that tokenizer compatibility is necessary but not sufficient — the draft model must also match the target model's output distribution, which requires training data from the specific model.

The throughput gain column was also carefully qualified. The AQ-MedAI EAGLE-3 model showed "~1.8x" based on published benchmarks. Training a new EAGLE-3 head showed "~1.5-2x expected" based on extrapolation from similar efforts. The proprietary Baseten model showed "~3x+" based on their blog post. These different levels of certainty were clearly signaled, demonstrating the assistant's commitment to distinguishing between measured results and expected outcomes.

Key Findings and Recommendations

The assistant's key findings were presented as a set of actionable conclusions. First, no official draft model exists for Kimi-K2.5 from Moonshot AI, and the model lacks native MTP capability. Second, DeepSeek V3's tokenizer is incompatible, ruling out what would otherwise be the most obvious source of draft models. Third, Moonshot AI has no small models in the Kimi family — their releases are all 1T parameter MoE models, with the only small model being Kimi-Audio-7B, which uses a different architecture entirely.

The recommendations were structured by use case. The best immediate option was AQ-MedAI/Kimi-K2-Instruct-eagle3 via SGLang, which was built for K2-Instruct and should partially transfer to K2.5. The best long-term option was training a K2.5-specific EAGLE-3 head using SpecForge, exactly as Baseten did for their 340+ TPS deployment. For llama.cpp users, jukofyork/Kimi-K2-Instruct-DRAFT-0.6B-v3.0-GGUF was the only available option.

Lessons in AI-Assisted Research

This subagent session offers several lessons about how AI-assisted research works in practice. First, the most important discovery was a negative finding. The tokenizer incompatibility between Kimi-K2.5 and DeepSeek V3 eliminated what would otherwise have been the most promising class of candidates. Without this finding, the user might have wasted significant effort trying to adapt DeepSeek V3's draft models or MTP module. The assistant's systematic approach to verifying this constraint — going directly to the source configuration files rather than relying on secondary sources — was methodologically sound.

Second, the research process was iterative and hypothesis-driven. Each round of tool calls built on knowledge accumulated in previous rounds, narrowing the search space and refining the questions being asked. The assistant moved from broad exploration (messages 1-2) to targeted verification (messages 3-6) to final gap-filling (message 7) to synthesis (message 8). This is a fundamentally scientific approach to the problem.

Third, the assistant demonstrated sophisticated use of parallel tool calls. By launching independent searches and fetches simultaneously, the assistant minimized latency and maximized information throughput. The decision to parallelize reflected an understanding of the dependency graph of the research questions — which queries were independent and which required results from others.

Fourth, the assistant handled uncertainty appropriately. Throughout the analysis, the assistant was careful to distinguish between what was known (tokenizer sizes, model architectures, framework requirements) and what was uncertain (transfer performance, acceptance rates, K2.5-specific behavior). The recommendations were presented with appropriate caveats, and the assistant explicitly flagged where empirical validation would be needed.

Conclusion

The nine-message subagent session examined in this article represents a complete research workflow compressed into a single conversation. Starting from a well-defined user request, the assistant systematically searched for candidate draft models, verified the critical tokenizer constraint through primary-source evidence, organized findings into a structured taxonomy, and delivered actionable recommendations with appropriate caveats.

The investigation revealed that the path to speculative decoding for Kimi-K2.5 is neither simple nor straightforward. No off-the-shelf draft model exists for K2.5 specifically. The most promising immediate option — the AQ-MedAI EAGLE-3 head trained for K2-Instruct — may partially transfer but requires empirical validation. The best long-term option — training a K2.5-specific EAGLE-3 head via SpecForge — requires significant investment but follows a proven methodology demonstrated by Baseten's production deployment.

For anyone engaged in speculative decoding research, the central lesson is clear: always verify the tokenizer. It is the silent constraint that can make or break an otherwise promising approach. And when the tokenizer doesn't match, the path forward requires either vocabulary transplantation or training a new draft model from scratch — both significant undertakings, but both now well-documented and reproducible thanks to tools like transplant-vocab and SpecForge. The draft model hunt for Kimi-K2.5 may not have yielded a perfect off-the-shelf solution, but it produced something almost as valuable: a clear map of the landscape, a structured decision framework, and a proven methodology for moving forward.## References

[1] "The Draft Model Hunt: Deconstructing a Speculative Decoding Research Request" — Analysis of the user's initial research request (msg id=0)

[2] "The Parallel Search: How an AI Agent Began Hunting for Draft Models for Kimi-K2.5 Speculative Decoding" — Analysis of the assistant's first response with parallel searches (msg id=1)

[3] "The Critical Fork: Investigating SpecBundle and Tokenizer Compatibility for Kimi-K2.5 Speculative Decoding" — Analysis of the focused follow-up investigation (msg id=4)

[4] "The Pivotal Verification: Settling the Tokenizer Question for Kimi-K2.5 Speculative Decoding" — Analysis of the config.json verification step (msg id=6)

[5] "The Pivot: From Data Gathering to Synthesis in the Search for Kimi-K2.5 Draft Models" — Analysis of the transition to synthesis (msg id=7)

[6] "The Crossroads of Speculation: Verifying Tokenizer Compatibility for Kimi-K2.5 Draft Models" — Analysis of the config.json fetch for Kimi-K2-Instruct (msg id=5)

[7] "The Hunt for Draft Models: Deepening the Investigation into Speculative Decoding for Kimi-K2.5" — Analysis of the deepening investigation (msg id=3)

[8] "The Tokenizer Trap: Why Finding a Draft Model for Kimi-K2.5 Requires Rethinking Speculative Decoding" — Analysis of the comprehensive synthesis message (msg id=8)

[9] "The Art of the Deep Dive: Orchestrating Speculative Decoding Research in a Single Message" — Analysis of the meta-research orchestration (msg id=2)