The Architecture Investigation: Systematic Repository Analysis of Kimi K2.5

Introduction

In the fast-paced world of large language model deployment, understanding a model's architecture is not merely an academic exercise — it is the foundation upon which every downstream decision about serving infrastructure, quantization strategy, speculative decoding integration, and performance optimization is built. When a team inherits a repository filled with benchmark scripts, configuration files, training plans, and patch notes, extracting a coherent architectural picture requires systematic, evidence-based investigation.

This article examines a complete sub-session within a broader opencode coding session, where an AI assistant was tasked with reverse-engineering the architecture of the Kimi K2.5 (and potentially K2.6) model from a local research repository. The investigation spans eight messages ([msg 0] through [msg 7]), representing a multi-phase research operation that moves from broad file discovery to targeted content search, deep configuration analysis, edge-case investigation, and ultimately a comprehensive findings report. This work exemplifies the kind of "repository archaeology" that is increasingly essential in modern ML engineering — the systematic extraction of knowledge from complex, multi-modal codebases.

The Mission: Five Critical Questions

The investigation was triggered by a precise user request ([msg 0]) that posed five architectural questions about the Kimi K2.5 or K2.6 model family. These questions were not random trivia; each targeted a specific constraint that would determine the feasibility and approach for the next phase of work — training a DFlash speculative decoding drafter for the model:

  1. Is it a pure attention model or does it have recurrent/Mamba/GDN layers? — This question directly reflected lessons learned from earlier benchmarks on the Qwen3.6 model, where "Mamba state leakage" in hybrid architectures had limited the effectiveness of DDTree speculative decoding. The user needed to confirm that Kimi K2.6 was pure attention before committing to the DDTree approach.
  2. What's the attention type (MHA, GQA, MLA)? — Multi-Head Attention, Grouped-Query Attention, and Multi-Head Latent Attention have vastly different memory footprints and computational profiles. MLA, for instance, compresses the KV cache into a latent space, dramatically reducing memory consumption for long-context inference.
  3. Is it MoE or dense? — Mixture-of-Experts models activate only a subset of parameters per token, changing memory partitioning strategies and communication patterns during tensor-parallel inference.
  4. What's the model size and layer count? — The most basic architectural parameters needed to design any drafter network, determine memory budgeting, and plan tensor parallelism configuration.
  5. Any mentions of speculative decoding or DFlash support for Kimi models? — A practical check to determine whether existing work could be leveraged rather than starting from scratch. The user directed the assistant to search a specific repository at /home/theuser/glm-kimi-sm120-rtx6000bw, which contained research artifacts from deploying Kimi models on 8× NVIDIA RTX PRO 6000 Blackwell GPUs.

Phase 1: Mapping the Terrain (Messages 1–2)

The assistant's first response ([msg 1]) was a broad reconnaissance operation. Rather than immediately searching for specific terms, the assistant ran recursive glob searches across the repository for all relevant file types: .md, .yaml, .yml, .json, .py, .sh, .toml, and .cfg. This "map the terrain before diving in" strategy is a hallmark of systematic investigation — you cannot search effectively until you know what you are searching through.

The glob results revealed a rich repository structure: benchmark result files (bench_results/), a benchmark runner script (bench_runner.py), a report generator (gen_report.py), a speculators subdirectory with its own documentation and CI configuration, vendored SGLang and PyTorch source trees, model configuration files, and training scripts. Critically, the globs revealed the existence of configs/kimi-k25-config.json — the model's configuration file — and ct129_sglang_full/sglang/srt/configs/kimi_linear.py — a hint at a separate hybrid model architecture that would later prove important.

In [msg 2], the assistant transitioned from inventory to targeted search. It ran a grep for the pattern (?i)(kimi|k2\\.5|k2\\.6|moonshot|moonshotai) — a well-constructed regex covering the model family name, version strings, and provider names. The search returned hundreds of matches across multiple files, with the densest concentrations in eagle-fast-verify.md (143 matches), gen_report.py (343 matches), configs/kimi-k25-config.json (7 matches), and eagle3-train/run_pipeline.sh (6 matches).

The assistant then performed a "triage read" of the four files the user had explicitly mentioned: k25b6000bench1.md, glm5findings.md, FINDINGS.md, and README.md. From these headers, the assistant extracted critical metadata: the model was "moonshotai/Kimi-K2.5 (INT4, compressed-tensors, W4A16, group_size=32)" running on "8x NVIDIA RTX PRO 6000 Blackwell Server Edition (SM120, 96GB GDDR7 each)" with "vLLM 0.16.0rc2.dev344 (V1 engine)" and "TP=8, max_model_len=131072." The README confirmed the repository contained "research artifacts, patches, and benchmark results from deploying Kimi K2.5 and K2-Thinking on 8x NVIDIA RTX PRO 6000 Blackwell (SM120) GPUs."

Phase 2: Deep Dive into Configuration (Message 3)

With the terrain mapped and the key files identified, the assistant moved to the most authoritative source of architectural truth: the model configuration file itself ([msg 3]). The assistant read kimi-k25-config.json alongside two EAGLE-3 planning documents (eagle-k2finetune-game-plan.md and next-steps-eagle.md).

The config JSON revealed the model's architecture name as KimiK25ForConditionalGeneration, with an auto-map pointing to configuration_kimi_k25.KimiK25Config and modeling_kimi_k25.KimiK25ForConditionalGeneration. The planning documents contained a critical revelation: "both models share the same DeepSeek V3 / MLA architecture, hidden_size, and layer structure." This directly answered the attention type question — the model uses Multi-Latent Attention (MLA), the same mechanism introduced in DeepSeek-V2/V3.

However, the config file content was truncated in the display, showing only the first 12 lines. The assistant could not yet see the critical parameters: num_hidden_layers, num_attention_heads, num_key_value_heads, hidden_size, intermediate_size, and the MoE configuration. These would require additional reads in subsequent messages.

The next-steps-eagle.md document provided context for speculative decoding: Baseten had achieved "340+ tok/s on Kimi-K2.5 using a custom EAGLE-3 speculator," compared to the current setup's 82.5 tok/s. The document noted that speculative decoding was "primarily a latency optimization" on the PCIe-only multi-GPU setup, where the verify forward pass was "almost entirely communication-bound."

Phase 3: The Pivot Point — DFlash and KimiLinear (Message 4)

Message [msg 4] marks a critical inflection point in the investigation. The assistant recognized that two questions remained partially unresolved: the recurrent/Mamba/GDN question and the DFlash question. It formulated a targeted grep pattern:

(?i)(dflash|kimi.*linear|kimi.*recurrent|kimi.*mamba|kimi.*gdn|KimiLinear)

This pattern was carefully constructed to test specific architectural hypotheses. Each term targeted a different possible mechanism: dflash for the diffusion-based speculative decoding method, kimi.*linear for linear attention layers, kimi.*recurrent for recurrent layers, kimi.*mamba for Mamba state space model layers, kimi.*gdn for a custom layer type, and KimiLinear for the exact class name.

The result was staggering: 8,798 matches across the repository. This volume indicated that DFlash and KimiLinear were deeply embedded throughout the codebase, appearing in benchmark reports, training scripts, configuration files, and documentation.

The assistant read kimi_linear.py from the SGLang source tree, which revealed a critical detail: the file imported from mamba_utils — specifically KimiLinearCacheParams and KimiLinearStateShape. This established a direct connection between KimiLinear layers and the Mamba state space model architecture. The presence of cache parameters and state shapes suggested that these layers maintain a recurrent state — characteristic of state space model layers, not standard Transformer attention layers.

This was a significant finding. The repository contained evidence of a separate model architecture called KimiLinearForCausalLM that uses hybrid attention-recurrent layers. But was this the same as Kimi K2.5?

The assistant also read dflash.md from the speculators documentation, which described DFlash as "a speculative decoding method that uses a small diffusion-LLM draft model to predict an entire block of tokens in a single forward pass." The documentation explicitly contrasted DFlash's diffusion-based approach with EAGLE-3's autoregressive approach.

Phase 4: Cross-Referencing and Resolution (Messages 5–6)

In [msg 5], the assistant cross-referenced the DDTree roadmap and benchmark plan to check for Kimi-specific DFlash mentions. The finding was clear: DFlash and DDTree were being developed and benchmarked for Qwen3.6-27B, not for Kimi models. The roadmap's goal was to deploy native DDTree for Qwen3.6 with the z-lab DFlash drafter. No Kimi-specific DFlash work was documented.

Message [msg 6] deepened the investigation into the KimiLinear architecture. The assistant re-read eagle-fast-verify.md for additional architectural details and ran a targeted grep for KimiLinearForCausalLM, kimi_linear, kda_layers, and linear_attn_config. The results confirmed that KimiLinearConfig was imported in SGLang's configuration infrastructure and referenced in model_config.py via a conditional check: elif "KimiLinearForCausalLM" in self.hf_config.architectures:. This suggested that when a model's HuggingFace config lists KimiLinearForCausalLM as its architecture, SGLang applies different handling — likely for a model variant that includes linear/recurrent layers.

Crucially, none of the visible matches connected KimiLinear to the K2.5 model specifically. The matches were all in SGLang's generic model configuration infrastructure, not in the K2.5-specific config file. This created a clear picture: KimiLinear is a separate model architecture from KimiK25.

Phase 5: The Comprehensive Findings Report (Message 7)

Message [msg 7] represents the culmination of the investigation — a structured, evidence-based findings report that answers all five user questions with supporting evidence from the repository.

Finding 1: Pure Attention Model

Kimi K2.5 is a pure attention model with NO recurrent, Mamba, or GDN layers. The config file shows the text backbone is DeepseekV3ForCausalLM — a pure transformer architecture. There is no linear_attn_config, no kda_layers, and no mamba/GDN configuration. The separate KimiLinearForCausalLM model that does have hybrid layers is a distinct architecture, not K2.5.

Finding 2: Multi-Latent Attention (MLA)

The config file contains the hallmark parameters of MLA: q_lora_rank: 1536, kv_lora_rank: 512, qk_nope_head_dim: 128, qk_rope_head_dim: 64, v_head_dim: 128, num_attention_heads: 64, num_key_value_heads: 64. These are the same parameters used in DeepSeek-V2/V3's MLA implementation, which compresses keys and values into a low-rank latent space to reduce KV cache memory.

Finding 3: MoE Architecture

Kimi K2.5 is a massive Mixture-of-Experts model with 384 routed experts, 8 experts active per token (plus 1 shared expert), an MoE intermediate size of 2048, and a sigmoid-based scoring function with noaux_tc top-k method. Only layer 0 is dense; layers 1–60 are all MoE.

Finding 4: Model Size and Layer Count

The model has 61 hidden layers, a hidden size of 7168, an intermediate size of 18432 for dense layers, approximately 1 trillion total parameters, and approximately 32 billion active parameters per token. The INT4 quantized version uses compressed-tensors with symmetric quantization, group_size=32, applied to routed MoE experts only (attention weights and shared experts remain in BF16).

Finding 5: Speculative Decoding Support

EAGLE-3 is the primary speculative decoding approach for Kimi models, with multiple planning documents, training scripts, and optimization work documented in the repository. An existing EAGLE-3 drafter exists for Kimi-K2 at AQ-MedAI/Kimi-K2-Instruct-eagle3 with acceptance lengths of 3.0–3.55. However, on the 8× RTX PRO 6000 PCIe setup, EAGLE-3 was found to be net-negative due to the ~30ms verify cycle time (97% spent on target model forward pass, dominated by AllReduce latency).

DFlash, by contrast, is documented and benchmarked for Qwen3.6-27B, not for Kimi models. No pretrained DFlash model for Kimi K2.5 is mentioned in the repository.

The report also delivered an important negative finding: there are no references to "K2.6" anywhere in the repository. All findings pertain to Kimi K2.5 (and some to Kimi K2 / K2-Thinking / K2-Instruct).

Methodology: What Made This Investigation Effective

Several methodological principles contributed to the success of this investigation:

1. Phased narrowing of scope. The assistant moved from broad file discovery (glob) to targeted content search (grep) to deep reading (config files) to edge-case investigation (KimiLinear). Each phase built on the previous one, ensuring that no relevant information was missed while avoiding premature conclusions.

2. Prioritization of configuration files as ground truth. The assistant consistently treated the model configuration JSON as the most authoritative source, using documentation files as corroborating evidence rather than primary sources. This is the right approach — configuration files contain the exact parameters used to instantiate the model, while documentation may contain errors or approximations.

3. Careful handling of ambiguous evidence. The discovery of KimiLinearForCausalLM — a model that does have recurrent/linear attention layers — presented an ambiguity that could have led to an incorrect conclusion. The assistant resolved this by reading both the KimiLinear configuration and the K2.5 configuration, comparing them, and explicitly stating that they are separate architectures.

4. Transparency and auditability. Every search result, file read, and intermediate finding was displayed inline, allowing the user to verify the assistant's work and catch any errors. This transparency is a key feature of the opencode paradigm — the assistant is not a black box producing answers, but a collaborator whose reasoning is visible.

5. Explicit negative findings. The report explicitly stated what was NOT found: no K2.6 references, no DFlash for Kimi, no recurrent layers in K2.5. Negative findings are often as valuable as positive ones — they prevent wasted effort pursuing blind alleys.

Broader Significance

This sub-session exemplifies a critical skill in modern ML engineering: the ability to navigate complex, multi-modal codebases and extract actionable architectural knowledge. As models grow larger and repositories more complex, this skill becomes ever more valuable. The assistant's approach — systematic, evidence-based, and clearly communicated — serves as a template for anyone facing a similar investigative task.

The investigation also illustrates the power of the subagent pattern in AI-assisted coding. By delegating the focused research task to a subagent that runs to completion before returning results, the main conversation can proceed efficiently without the overhead of iterative Q&A. The subagent's comprehensive report provides the architectural foundation needed for the next phase of work — whether that involves training a DFlash drafter, deploying EAGLE-3 speculative decoding, or exploring alternative acceleration methods.

For the broader project of deploying Kimi K2.5 on Blackwell GPUs, the architectural knowledge gathered in this sub-session is foundational. Knowing that the model uses MLA attention informs KV cache memory calculations. Knowing it is MoE with 61 layers informs memory partitioning and tensor parallelism configuration. Knowing that EAGLE-3 has been extensively tested (and found net-negative on this hardware) informs decisions about whether to pursue speculative decoding further. And knowing that DFlash has not been explored for Kimi models opens the door for future work — which is precisely where the conversation pivots next.