The Weight Key Remapping Problem: Bridging Pretrained Checkpoints in EAGLE-3 Finetuning

In the midst of a multi-hour inference run generating 25,000 synthetic reasoning samples from the Kimi-K2.5 model, the assistant in this opencode session took a strategic pause to prepare the next phase of the pipeline. Message [msg 2883] captures a moment of careful architectural reasoning: the assistant has just downloaded a pretrained EAGLE-3 draft model checkpoint from AQ-MedAI, verified its weight keys, and now must figure out how to integrate it into their training pipeline. The message is short — a few lines of analysis and a single read tool call — but it encapsulates a critical design decision that bridges two separate training approaches.

The Context: From Random Init to Finetuning

The broader session had been building an EAGLE-3 speculative decoding system for Kimi-K2.5, a 1-trillion-parameter Mixture-of-Experts model running on 8× NVIDIA RTX PRO 6000 Blackwell GPUs. Earlier in segment 22 (see [chunk 22.0]), the assistant had successfully implemented the core EAGLE-3 training pipeline from scratch: writing 04_train.py to use the speculators library's Eagle3SpeculatorConfig, Eagle3DraftModel, and built-in Trainer class, then validating it on 10 samples and scaling to 1,000 samples. That pipeline trained the draft model with randomly initialized weights — a perfectly valid approach, but one that starts from zero knowledge.

The user then redirected the strategy: instead of training from random initialization, they wanted to generate high-quality synthetic training data by capturing Kimi-K2.5's actual reasoning outputs via the vLLM inference server. The assistant wrote 01b_generate_synthetic.py, fixed timeout and field-extraction bugs, and launched a 25K-sample generation run ([msg 2876]). While that inference job churned away (expected to take ~5 hours), the assistant downloaded the AQ-MedAI/Kimi-K2-Instruct-eagle3 checkpoint — a pretrained EAGLE-3 drafter for the original Kimi-K2 model — with the intention of using it as a finetuning starting point ([msg 2874], [msg 2879]).

Message [msg 2883] is the moment where these two threads converge. The download has completed ([msg 2881]), the weight keys have been inspected, and the assistant now faces the integration problem.

The Core Problem: midlayer.* vs layers.0.*

The message opens with a clear statement of the discovered incompatibility:

Now the AQ-MedAI model uses midlayer.* keys while our model uses layers.0.*. For finetuning, I need to: 1. Load the AQ-MedAI weights 2. Remap midlayer.* → layers.0.* 3. Load into our Eagle3DraftModel 4. The d2t/t2d from AQ-MedAI might differ from ours (they trained on K2, different token frequencies) — we should rebuild them from our data

This is a deceptively simple observation that reveals several layers of understanding about the system's architecture.

Why the key names differ: The EAGLE-3 draft model is a single transformer layer (the "midlayer" in AQ-MedAI's terminology) that predicts hidden states from the verifier model's representations. AQ-MedAI, who trained their drafter on the original Kimi-K2, named this layer midlayer in their checkpoint. The assistant's own pipeline, built using the speculators library's Eagle3DraftModel class, names it layers.0 — following the Hugging Face/transformers convention where transformer layers are indexed numerically. Both names refer to the same thing: the single transformer block that forms the core of the EAGLE-3 draft model. But the string difference means that a naive torch.load() followed by model.load_state_dict() would fail with a key mismatch error.

The remapping strategy: The assistant's plan is straightforward — a simple string substitution of midlayer → layers.0 when loading the state dictionary. This is a mechanical operation, but it's one that requires precise knowledge of both naming conventions. The assistant had to have inspected the AQ-MedAI checkpoint's keys (done in [msg 2881]) and compared them against the keys produced by their own Eagle3DraftModel instantiation. This cross-referencing is the kind of careful detective work that distinguishes a working integration from a broken one.

The d2t/t2d Decision: Why Rebuild From Scratch

The fourth point in the assistant's plan is particularly insightful: "The d2t/t2d from AQ-MedAI might differ from ours (they trained on K2, different token frequencies) — we should rebuild them from our data."

The d2t (decision-to-token) and t2d (token-to-decision) mappings are auxiliary components of the EAGLE-3 architecture. They map between the draft model's "decision" tokens (which represent the model's internal prediction states) and the verifier model's vocabulary tokens. These mappings are learned from data — specifically, from the token frequency distribution of the training corpus.

AQ-MedAI trained their drafter on the original Kimi-K2, which uses the same base vocabulary but may have different token frequency distributions than Kimi-K2.5. More importantly, the assistant's training data comes from open-perfectblend (a dataset of reasoning questions) processed through Kimi-K2.5's actual inference outputs. The token distribution of these synthetic reasoning traces will differ substantially from whatever corpus AQ-MedAI used. Rebuilding the d2t/t2d mappings from the actual training data ensures that the draft model's decision space aligns with the token patterns it will actually encounter during speculative decoding.

This decision reflects a nuanced understanding of what parts of a pretrained checkpoint can be safely transferred and what parts must be recomputed for the new domain. The transformer weights (the midlayer/layers.0 parameters) capture generalizable knowledge about how to predict hidden states — these are worth transferring. The d2t/t2d mappings, by contrast, are dataset-specific statistics that should be derived from the actual training distribution.

The --finetune-from Design

The assistant announces the plan to "update 04_train.py to support a --finetune-from argument" and immediately issues a read tool call to examine the current file. This is a deliberate, structured approach to code modification: understand the existing code before making changes.

The read tool returns the first 10 lines of 04_train.py, showing its docstring and the beginning of the file. The assistant is loading the full file into context to understand the current training loop structure, where the model is instantiated, and where checkpoint loading would need to be inserted. The --finetune-from argument will serve as a command-line flag that, when provided, triggers the weight loading and remapping logic instead of random initialization.

This design choice — a command-line flag rather than a separate script or configuration file — keeps the training pipeline unified. The same 04_train.py can handle both training from scratch (when --finetune-from is absent) and finetuning from a pretrained checkpoint (when it's provided). This is good software engineering: it reduces code duplication, maintains a single entry point for training, and makes the finetuning path testable alongside the scratch path.

Assumptions and Potential Pitfalls

The assistant makes several assumptions in this message, most of which are reasonable but worth examining:

  1. Architectural compatibility: The assumption that AQ-MedAI's EAGLE-3 drafter uses the same architecture as the assistant's Eagle3DraftModel (same layer count, same hidden dimension, same activation function, etc.). This was partially verified in [msg 2881] where the weight shapes were inspected — fc.weight: torch.Size([7168, 21504]) matches the expected dimensions for a single-layer transformer with hidden size 7168 and intermediate size 21504. But shape compatibility doesn't guarantee architectural compatibility (e.g., different normalization placement, different activation functions).
  2. Simple key remapping: The assumption that midlayer.* → layers.0.* is a complete and sufficient remapping. If the AQ-MedAI checkpoint contains additional keys that don't exist in the assistant's model (or vice versa), the load_state_dict call with strict=True would fail. The assistant would need to handle this gracefully, perhaps with strict=False and logging of unmatched keys.
  3. Random init for missing layers: The assumption that any layers not present in the AQ-MedAI checkpoint (e.g., if the assistant's model has additional components) should be randomly initialized. This is the standard approach for partial checkpoint loading.
  4. The d2t/t2d can be rebuilt: The assumption that rebuilding d2t/t2d from the training data is both feasible and beneficial. This is almost certainly correct, but it adds complexity to the training pipeline — the script must now handle the case where these mappings are computed from data rather than loaded from a checkpoint.

Input and Output Knowledge

Input knowledge required to understand this message includes:

The Thinking Process

The message reveals the assistant's thinking process at multiple levels. At the surface level, it's a straightforward planning statement: "Here's the problem, here's my plan, let me start implementing." But beneath that, several cognitive processes are visible:

Parallel task management: The assistant is running a long inference job (25K samples, ~5 hours) and using that time productively to prepare the next pipeline stage. This is efficient orchestration of sequential dependencies — the inference must complete before training can begin, but the training script can be prepared in advance.

Comparative analysis: The assistant has just inspected the AQ-MedAI checkpoint's weight keys ([msg 2881]) and is comparing them against the keys produced by their own Eagle3DraftModel. This cross-referencing is a form of differential analysis — identifying what's the same (the transformer layer weights) and what's different (the key names, the d2t/t2d mappings).

Transfer learning reasoning: The assistant is making a nuanced decision about what to transfer from the pretrained checkpoint and what to recompute. The transformer weights (generalizable knowledge) are transferred; the d2t/t2d mappings (dataset-specific statistics) are recomputed. This reflects an understanding of which parts of the model capture domain-general vs. domain-specific information.

Error anticipation: By planning the key remapping and the d2t/t2d rebuild explicitly, the assistant is anticipating potential failure modes. A naive approach — just loading the entire checkpoint without remapping — would crash. A slightly less naive approach — loading with strict=False — might silently drop or misalign weights. The explicit plan avoids both pitfalls.

Conclusion

Message [msg 2883] is a small but pivotal moment in a much larger engineering effort. It's the bridge between two training strategies: from-scratch random initialization and pretrained-checkpoint finetuning. The assistant's analysis of the midlayer.* vs layers.0.* key mismatch, the decision to rebuild d2t/t2d mappings, and the structured approach to modifying the training script all demonstrate a careful, methodical engineering mindset. The message itself is brief — a few lines of reasoning and a tool call — but it encapsulates the kind of architectural detective work that makes the difference between a pipeline that crashes on the first checkpoint load and one that smoothly integrates pretrained knowledge into a custom training workflow.