Segment 30

This sub-session completed the EAGLE-3 training pipeline from data preparation through deployment. Hidden state extraction finished successfully with 37,312 samples (87.8M tokens, ~4.6 TB) and zero errors. Training on 4 GPUs with TTT=5, batch_size=8, and max_seq_len=8192 achieved ~100% GPU utilization and completed 5 epochs in ~10.8 hours, converging to 74.7% validation accuracy and an estimated acceptance length of ~2.95 tokens — a significant improvement over the previous 10K drafter's 2.1. Along the way, several technical challenges were resolved: fixing Triton shared-memory OOM by reducing sequence length and using batch packing, correcting SGLang server argument names for speculative decoding, and applying weight key fixes for SGLang compatibility. The drafter checkpoint was prepared with vLLM-compatible config, and the SGLang server was deployed with EAGLE3 speculation at 16 draft tokens, ready for benchmarking.

Complete hidden state extraction for 100K datasetTrain EAGLE-3 draft model on 100K samplesFix Triton shared-memory OOM during trainingCorrect SGLang speculative decoding argument namesApply weight key fix for SGLang compatibilityDeploy EAGLE-3 draft model with SGLang speculationPrepare vLLM-compatible drafter checkpoint

The EAGLE-3 Pipeline Comes Together: From 100K Hidden States to Deployed Speculation on Kimi-K2.5 3971 words

Chunks