The Moment of Truth: First Execution of the EAGLE-3 Training Pipeline for Kimi-K2.5

In any complex engineering effort, there comes a pivotal moment when all the preparatory work—the research, the code exploration, the compatibility patches, the data pipeline construction—converges into a single command. Message 2758 of this opencode session is exactly that moment. After an extensive journey through the speculators library's training API, multiple rounds of monkey-patching to accommodate Kimi-K2.5's unusual nested configuration structure, and a complete rewrite of the training script, the assistant finally launches the first test run of the EAGLE-3 draft model training pipeline.

The message itself is deceptively simple. It begins with a brief confirmation—"Good, all GPUs are free"—followed by a single bash command that executes the newly rewritten 04_train.py on the remote container, and then the first lines of output streaming back. But behind this simple exchange lies the resolution of a cascade of technical challenges that had occupied the previous fifteen messages of the conversation.

The Motivation: Why This Message Was Written

The immediate trigger for this message was the successful completion of the 04_train.py rewrite. In the messages leading up to this point ([msg 2739] through [msg 2755]), the assistant had conducted a deep investigation of the speculators library's training infrastructure. This investigation revealed two critical incompatibilities between speculators and the Kimi-K2.5 model architecture.

First, speculators' Eagle3DraftModel._setup_embeddings_and_lm_heads method calls AutoConfig.from_pretrained() on the verifier model path and then accesses verifier_model_config.hidden_size directly. But Kimi-K2.5 uses a multimodal wrapper architecture: the top-level config is a KimiK25Config that contains a nested text_config of type DeepseekV3Config. The hidden_size attribute (value 7168) lives on the nested config, not the top level. The speculators code would crash with an AttributeError when it tried to access it.

Second, the weight key names in Kimi-K2.5's safetensors index are prefixed with language_model.—so the embedding weight is at language_model.model.embed_tokens.weight and the LM head at language_model.lm_head.weight. Speculators hardcodes the unprefixed paths model.embed_tokens.weight and lm_head.weight, which would cause weight loading to fail silently or load the wrong tensors.

The assistant considered two approaches: patching speculators' source code directly, or monkey-patching the method at runtime within the training script. The assistant chose the latter, reasoning it was "cleaner for a training script" ([msg 2752]). This decision reflects a pragmatic engineering judgment: modifying the installed library would create maintenance burden and potentially break other functionality, while a targeted monkey-patch keeps the customization contained within the training pipeline itself.

The Decision-Making Process

The training parameters chosen in this message reveal the assistant's strategic thinking. The test uses a single GPU (CUDA_VISIBLE_DEVICES=0), only 3 epochs, a modest learning rate of 3e-5, and zero data-loading workers. These are deliberately conservative choices designed to minimize complexity and maximize the chances of a clean first run. The assistant is not trying to achieve good training results yet—it is validating that the entire pipeline works end-to-end.

The choice of --num-workers 0 is particularly telling. In PyTorch data loading, multi-process workers introduce complexity around shared memory, CUDA tensor pinning, and error propagation. By setting workers to zero, the assistant ensures that any error will be immediately visible in the main process traceback, without the indirection of worker process crashes. This is a debugging-first mindset.

The 20% validation split (--val-ratio 0.2) with only 10 total samples means the validation set is just 2 samples—statistically meaningless but sufficient to confirm that the validation loop code path executes without errors. The 3 TTT steps (Test-Time Training steps) match the EAGLE-3 architecture's requirement for multi-step token prediction during training.

The Assumptions Embedded in This Run

This first execution rests on several assumptions that the assistant made during the preceding investigation. One key assumption is that the vocabulary mapping tensors (t2d.pt and d2t.pt) built by the custom 03_build_vocab_mapping.py script are compatible with speculators' expectations. The assistant verified this by inspecting the tensor shapes and confirming they match the speculators convention: t2d is a boolean mask of shape [163840] (target vocabulary size) with exactly 32,000 True entries (draft vocabulary size), and d2t is an index mapping of shape [32000].

Another assumption is that the hidden state data format (v1) produced by the extraction step is correctly interpreted by speculators' Eagle3SampleFileDataset and standardize_data_v1 functions. The assistant checked this by loading a sample file and confirming it has the expected keys (input_ids, hidden_states, loss_mask) with the correct shapes and dtypes.

A third assumption—which would prove partially incorrect—is that the noise transform API (AddUniformNoise, TransformTensors) would work as expected with the training script. In the very next message ([msg 2759]), the assistant discovers that TransformTensors.__init__ takes positional arguments (std, tensors) rather than keyword arguments, requiring a fix. This is a classic example of the kind of API mismatch that emerges when using a library without thorough documentation—the assistant had inspected the class but missed the constructor signature nuance.

The Knowledge Required to Understand This Message

To fully grasp what is happening in this message, one must understand the layered architecture of both the model and the training framework. Kimi-K2.5 is a 1-trillion-parameter Mixture-of-Experts model with a DeepSeek V3 / MLA (Multi-head Latent Attention) architecture. It has 61 layers, 384 routed experts with top-8 selection, and a vocabulary of 163,840 tokens. The model is quantized to INT4 using the compressed-tensors format with Marlin W4A16 kernels for the MoE layers.

The EAGLE-3 draft model being trained is a single-layer transformer with hidden size 7168, 64 attention heads, and a reduced vocabulary of 32,000 tokens. It takes auxiliary hidden states from layers 2, 30, and 58 of the main model as conditioning signals, and learns to predict the next token autoregressively using a technique called Test-Time Training (TTT). The draft model's smaller vocabulary is mapped from the full 163,840-token vocabulary using frequency-based selection—the most common 32,000 tokens in the training data become the draft vocabulary.

The speculators library (v0.3.0) provides the training infrastructure: Eagle3SpeculatorConfig for configuration, Eagle3DraftModel for the model architecture, and a Trainer class that handles the training loop, validation, checkpointing, and logging. The library was designed for standard HuggingFace model architectures and required extensive patching to work with Kimi-K2.5's nested config structure and vLLM 0.16's API changes.

What This Message Creates

The output shown in this message is the first evidence that the entire EAGLE-3 training pipeline is functional. The model loaded successfully, the speculators config was parsed, and training began. The benign warning about MLPSpeculatorConfig shadowing an attribute is a known issue in the speculators library and does not affect functionality.

This message creates several forms of knowledge. It confirms that the monkey-patch approach for verifier weight extraction works—the model was able to load the embedding and LM head weights from the Kimi-K2.5 checkpoint despite the nested config structure and non-standard weight key prefixes. It validates that the data pipeline (hidden state extraction → vocabulary mapping → training) is correctly wired end-to-end. And it establishes a baseline for training performance that will be measured and optimized in subsequent runs.

Perhaps most importantly, this message marks the transition from preparation to execution. The assistant had spent many messages exploring, patching, and rewriting. Now, for the first time, the code is running on real hardware with real data. The training has begun.