The Moment of Truth: EAGLE-3 Training Begins at Scale

In the long arc of an opencode coding session, most messages are about doing — executing commands, debugging errors, rewriting code. But some messages are about witnessing. Message 2824 falls into the latter category. It is a brief status check, a single tail -30 command piped through a 30-second sleep, but what it captures is the culmination of hours of painstaking engineering: the EAGLE-3 draft model training pipeline, running for the first time at meaningful scale on 1000 real samples.

The Message in Full

The message is deceptively simple. The assistant runs:

sleep 30 && ssh root@10.1.230.174 'tail -30 /root/eagle3-train/train_1k.log 2>/dev/null'

And receives back:

  d2t: torch.Size([32000])

Extracting verifier embedding + lm_head weights...
  Loading embed_tokens from model-00062-of-000064.safetensors (language_model.model.embed_tokens.weight)
    Shape: torch.Size([163840, 7168]), dtype: torch.bfloat16
  Loading lm_head from model-00062-of-000064.safetensors (language_model.lm_head.weight)
    Shape: torch.Size([163840, 7168]), dtype: torch.bfloat16

Building EAGLE-3 config...
Verifier: hidden_size=7168, vocab_size=163840

Creating EAGLE-3 draft model.....

These six lines of log output represent the successful crossing of a critical threshold. The training script has loaded its dependencies, initialized the vocab mapping, begun extracting the verifier model's embedding and language model head weights, and is now constructing the EAGLE-3 draft model architecture. The training loop itself has not yet begun — the message captures the initialization phase — but every line printed means that a potential failure mode has been avoided.

The Weight of Context

To understand why this message matters, one must appreciate the journey that led to it. The assistant had spent the preceding hours wrestling with the speculators library, an open-source package for implementing speculative decoding with EAGLE-style draft models. The library's API had proven incompatible with vLLM 0.16 in multiple ways: the KV cache config API had changed, the Scheduler and Request constructors had been refactored, and the custom worker for the DeepseekV2 architecture (which Kimi-K2.5 inherits) needed a complete rewrite of its forward pass (<msg id=2795-2798>). Each incompatibility had required careful study of the library's source code, followed by surgical monkey-patching or outright replacement of the offending functions.

The training pipeline itself — 04_train.py — had been rewritten from scratch after the assistant discovered that the speculators library provided its own Trainer class and Eagle3SpeculatorConfig that should be used instead of a hand-rolled training loop. The assistant had to understand the library's data flow: how Eagle3DraftModel wraps the verifier, how the TTT (Test-Time Training) objective works, how the vocab mapping bridges the 163,840-token verifier vocabulary to the 32,000-token draft vocabulary, and how the packing collator assembles variable-length sequences into efficient batches.

Before training could even begin, there was the ordeal of hidden state extraction. The 547 GB Kimi-K2.5 INT4 model had to be loaded across all 8 GPUs with tensor parallelism, a process taking 22.5 minutes. The first attempt failed with an out-of-memory error because the assistant had set --batch-size 2000, attempting to prefill all 503,000 tokens at once ([msg 2813]). After killing the processes, freeing the GPUs, and re-running with --batch-size 4, the extraction completed successfully: 1000 samples, 502,983 tokens, extracted at 2,912 tokens per second in under 3 minutes (<msg id=2818-2819>). The resulting 27 GB of hidden state data on disk represented the training foundation for the EAGLE-3 draft model.

What the Log Lines Reveal

Each line in the subject message's output tells a story:

d2t: torch.Size([32000]) — The draft-to-target ID mapping tensor has been loaded successfully. This tensor, built in step 3 of the pipeline, maps each of the 32,000 draft vocabulary tokens to their corresponding target vocabulary IDs. Its presence confirms that the vocab mapping pipeline ran correctly, achieving 100% coverage of the training data's token frequency.

Extracting verifier embedding + lm_head weights... — The training script is now loading parts of the verifier model (the full Kimi-K2.5) that are needed for the EAGLE-3 architecture. Specifically, it needs the input embedding matrix (embed_tokens) and the language model head (lm_head), which maps hidden states to vocabulary logits. These are loaded from the specific safetensor shard model-00062-of-000064.safetensors.

Shape: torch.Size([163840, 7168]), dtype: torch.bfloat16 — The embedding matrix is 163,840 rows (the verifier's full vocabulary) by 7,168 columns (the model's hidden dimension), stored in bfloat16 precision. This is a massive matrix — approximately 2.3 GB in memory — that the EAGLE-3 draft model will use as its own embedding layer, sharing the verifier's representation of input tokens.

Verifier: hidden_size=7168, vocab_size=163840 — The EAGLE-3 configuration is being built from the verifier's properties. The draft model will have the same hidden dimension but a much smaller vocabulary (32,000 vs 163,840), which is the core of the efficiency gain: the draft model only needs to predict tokens from a reduced vocabulary.

Creating EAGLE-3 draft model..... — The ellipsis here is telling. The draft model construction involves initializing the multi-layer transformer blocks (the "EAGLE-3 body"), the feature compression network (fc), the hidden norm layers, and the TTT (Test-Time Training) components. Each dot represents a stage of initialization completing without error.

Assumptions and Decisions

The assistant made several assumptions in this message that are worth examining. First, it assumed that a 30-second sleep was sufficient for the training script to reach the weight extraction phase. This was a reasonable guess based on the fact that the script had already loaded the vocab mapping (a small tensor operation) and was proceeding to load safetensor shards. In practice, the 30-second window was enough to confirm that the pipeline was alive and progressing, but not enough to see the first training step — that would require waiting another 2-3 minutes for the model construction to complete.

Second, the assistant assumed that the weight extraction logic — monkey-patched to handle Kimi-K2.5's nested configuration structure (where the verifier's config is wrapped inside a language_model key) — would work correctly. This was a validated assumption: the earlier 10-sample test run had already exercised this code path and succeeded.

Third, there was an implicit assumption that the training script would not encounter a new error at this stage. The pipeline had been tested on 10 samples, but scaling to 1000 samples could theoretically expose issues like memory fragmentation, data loading bottlenecks, or numerical instability. The assistant's decision to check after only 30 seconds reflects a desire for early validation — catch failures fast rather than wait minutes for a complete run.

Input Knowledge Required

To fully understand this message, one needs knowledge of the EAGLE-3 speculative decoding architecture: how a lightweight "draft" model is trained to predict the hidden states of a much larger "verifier" model, using a reduced vocabulary and a multi-layer feature prediction head. One also needs familiarity with the speculators library's API conventions — the Eagle3SpeculatorConfig, Eagle3DraftModel, and Trainer classes — and with the Kimi-K2.5 model's architecture, which inherits from DeepseekV2 with its MoE (Mixture of Experts) layers and MLA (Multi-head Latent Attention).

Additionally, understanding the safetensor shard naming convention (model-00062-of-000064.safetensors) requires knowledge that large models are split across multiple files for distributed loading, and that the embedding and lm_head weights happen to reside in shard 62 of 64. The hidden dimension of 7,168 and vocabulary size of 163,840 are specific to the Kimi-K2.5 model family.

Output Knowledge Created

This message creates knowledge about the state of the training pipeline at a specific point in time. It confirms that:

  1. The vocab mapping (d2t tensor) loaded correctly with the expected shape.
  2. The safetensor loading infrastructure can locate and load specific weight matrices from the multi-shard checkpoint.
  3. The weight extraction monkey-patch works — it correctly accesses language_model.model.embed_tokens.weight and language_model.lm_head.weight from the nested model structure.
  4. The EAGLE-3 configuration builder correctly infers hidden_size and vocab_size from the verifier.
  5. The draft model construction has begun without immediate errors. This output serves as a checkpoint for the assistant: the pipeline has survived the initialization phase, and the next status check (in message 2825) will reveal whether training steps are actually executing.

The Thinking Process

The assistant's reasoning in this message is visible in its choice of timing and what to check. The 30-second delay is deliberate — long enough for the script to progress past Python import time and initial logging, but short enough that if something fails immediately (e.g., a missing file or import error), the assistant can catch it quickly and intervene. The tail -30 command captures the last 30 lines of the log, which should include the most recent progress. The 2&gt;/dev/null redirect suppresses stderr in case the log file doesn't exist yet.

The assistant is operating in a "trust but verify" mode. It has already validated the pipeline on 10 samples, but scaling by two orders of magnitude (from 10 to 1000) introduces risk. Rather than waiting for the full training run to complete (which would take ~30 minutes for 10 epochs), it checks early to confirm the pipeline is alive. This is a standard pattern in the assistant's workflow: launch a long-running job, wait briefly, verify it started correctly, then let it run to completion with periodic monitoring.

Conclusion

Message 2824 is a quiet victory lap. After hours of debugging API incompatibilities, rewriting training scripts, recovering from OOM errors, and waiting through 22-minute model loads, the assistant finally sees the EAGLE-3 training pipeline initialize at scale. The log lines are mundane — tensor shapes, file paths, configuration values — but they represent the successful integration of dozens of components: the speculators library, the vLLM inference engine, the Kimi-K2.5 model architecture, the hidden state extraction pipeline, and the data preparation workflow. Each component had been a potential failure point, and each had been addressed in turn. The message captures the moment when all those components finally work together, and the real work — training a draft model that could double or triple inference throughput — can begin.