You can deploy this model and start earning money today!

XLOverflow/
qwen3-eagle3-baseline-uniform

Name: qwen3-eagle3-baseline-uniform
Availability: InStock

AngelSlim/Qwen3-8B_eagle3

finetune

apache-2.0

Views

No views yet

EAGLE-ft (Uniform) — EAGLE3 Draft Model for Qwen3-8B

EAGLE3 draft fine-tuned on our training data with uniform per-step loss weights. Baseline.

Part of a course project evaluating per-step weighted loss functions for training EAGLE3 draft models. Full pipeline and source: https://github.com/XLOverflow/anlp_course_project

Collection: Qwen3 EAGLE3 — Weighted Loss Variants

Training

Framework: SpecForge (our fork: https://github.com/XLOverflow/SpecForge)
Target model: Qwen/Qwen3-8B
Draft init: AngelSlim/Qwen3-8B_eagle3
Data: ShareGPT-style reasoning traces (see scripts/data/ in project repo)
Loss weight per step: uniform (w_s = 1)
Epochs: 5
LR: 3e-5
Batch size: 1
Max length: 4096

Evaluation (Qwen3-8B target)

Dataset	τ (accept. length)	Speedup	Accuracy
GSM8K	6.909	4.275×	95.60%
MATH500	6.758	4.245×	94.60%

Baselines for reference: Vanilla ≈ 1× speedup, EAGLE-orig ≈ 2× speedup.

Files

model.safetensors — draft model weights (~763 MB)
config.json — model config
Corresponds to: outputs/eagle3-baseline-uniform/epoch_4_step_82000 in the original training output

Optimizer state (~3 GB) is not uploaded — use the project repo's training scripts to resume from scratch if needed.

Usage

python

from huggingface_hub import snapshot_download
draft_path = snapshot_download(repo_id="XLOverflow/qwen3-eagle3-baseline-uniform")
# Then load with EAGLE's EaModel — see scripts/eval/eval_combined.py in the project repo.