You can deploy this model and start earning money today!

inference-optimization/
Qwen3-8B-speculators.peagle-qwen3arch-ckpt4

Name: Qwen3-8B-speculators.peagle-qwen3arch-ckpt4
Availability: InStock

Views

No views yet

Qwen3-8B P-EAGLE Drafter (Qwen3 Architecture)

P-EAGLE (Parallel Eagle) speculative decoding drafter for Qwen/Qwen3-8B using the Qwen3 draft architecture (--draft-arch qwen3).

The Qwen3 architecture uses additional q_norm/k_norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See speculators#563 for the RFC and experimental results.

Benchmark Results

Evaluated on RedHatAI/speculator_benchmarks (9 subsets) via vLLM throughput mode.

Acceptance lengths

Subset	k=5	k=7
HumanEval	3.51	3.73
Math Reasoning	3.73	4.04
QA	2.81	3.01
Question	2.99	3.27
RAG	3.06	2.98
Summarization	2.72	2.55
Tool Call	2.71	2.97
Translation	2.80	2.81
Writing	3.00	3.37
Average	3.04	3.19

Comparison vs Llama-arch baseline

Format: delta vs Llama-arch P-EAGLE.

Subset	k=5	k=7
HumanEval	+0.50	+0.23
Math Reasoning	+0.46	+0.22
QA	+0.41	+0.19
Question	+0.36	+0.24
RAG	+0.57	+0.05
Summarization	+0.69	+0.07
Tool Call	+0.26	+0.10
Translation	+0.54	-0.07
Writing	+0.37	+0.31
Average	+0.46 (+18%)	+0.15 (+5%)

Qwen3-arch outperforms Llama-arch on all 9 subsets at k=5 and 8/9 at k=7.

Training

Parameter	Value
Target model	Qwen/Qwen3-8B
Draft architecture	Qwen3 (`--draft-arch qwen3`)
Learning rate	6e-4
Epochs	5
Draft vocab size	32000
Sequence length	8192
Num layers	4
Num depths	7
Down-sample ratio	0.6 (min 0.2)
Scheduler	Cosine
Hardware	2x H200
Training library	speculators

Usage

Requires vLLM with Qwen3 Eagle3/P-EAGLE support (vllm#43132) and the architecture resolution fix.

bash

vllm serve Qwen/Qwen3-8B \
  --speculative-config '{
    "model": "inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4",
    "num_speculative_tokens": 7,
    "method": "eagle3",
    "parallel_drafting": true
  }'

speculators#563 — RFC: Support Qwen3 base architecture for Eagle3 & P-EAGLE
Llama-arch P-EAGLE baseline — Same setup with Llama draft architecture
Eagle3 Qwen3-arch drafter — Eagle3 (sequential) variant

inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4