Eagle3 speculative decoding drafter for
Qwen/Qwen3-8B using the
Qwen3 draft architecture (
--draft-arch qwen3).
The Qwen3 architecture uses additional
q_norm/
k_norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See
speculators#563 for the RFC and experimental results.
Evaluated with sampling params temp=0.6, top_p=0.95, top_k=20.
Trained on the same dataset with the same hyperparameters (except LR). Format: delta vs
llamaarch-ckpt1.
Outperforms the Llama-arch baseline by ~3-7% across all benchmarks and all k values, with the gap widening at higher k (math k=7: 3.68 vs 3.45, +6.7%).
1vllm serve Qwen/Qwen3-8B \
2 --speculative-config '{
3 "model": "inference-optimization/Qwen3-8B-from-Qwen3-8B_regen-speculators.eagle3-qwen3arch-ckpt1",
4 "num_speculative_tokens": 3,
5 "method": "eagle3"
6 }'
