Segment 31
In this sub-session, the user investigated why the newly trained EAGLE-3 draft model was not improving throughput despite 74.7% training accuracy. Two root causes were identified: first, `--speculative-num-steps 1` was silently overriding `--speculative-num-draft-tokens 16` to just 2 draft tokens; after fixing to 15 steps, performance was still poor at 46.7 tok/s. A standalone test revealed a critical wiring mismatch: training used `cat([embed_output, layer3, layer31])` as input to the draft model's fc layer, but SGLang was passing `cat([layer3, layer31, layer59])` (only auxiliary hidden states, missing the embedding output). The user modified `deepseek_v2.py` to capture the embedding output when `layer_id=-1` is specified, updated the draft model config from `[2, 30, 58]` to `[-1, 2, 30]`, and restarted the server. After the fix, performance improved slightly to 54.8 tok/s with accept length ~1.8 out of 6 draft tokens, still far below the 90.0 baseline, indicating additional issues remain.