File	Format	Size	Description
`eagle3_draft_f16.gguf`	GGUF F16	470 MB	Draft model in F16 precision
`eagle3_draft_q8.gguf`	GGUF Q8_0	238 MB	Draft model quantized to Q8_0
`hf_model/`	safetensors	896 MB	Hugging Face format for fine-tuning

Usage with llama.cpp

bash

# Basic usage with speculative decoding
llama-cli \
  -m Qwen3-0.6B-target.gguf \
  -md eagle3_draft_q8.gguf \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 5 \
  -p "Hello, how are you?" \
  -n 200

# Server mode (enables Ollama-like API)
llama-server \
  -m Qwen3-0.6B-target.gguf \
  -md eagle3_draft_q8.gguf \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 5 \
  --port 8080

Usage with Ollama

Pull the target model:

ollama pull qwen3:0.6b

Download the draft model GGUF from this repo
Create a Modelfile:

dockerfile

FROM qwen3:0.6b
DRAFT_MODEL ./eagle3_draft_q8.gguf
PARAMETER speculative_type draft-eagle3
PARAMETER speculative_draft_max 5

Create and run:

bash

ollama create qwen3-eagle -f Modelfile
ollama run qwen3-eagle

Benchmark (CPU-only, 16GB RAM)

Config	Speed (t/s)	vs Baseline
Baseline (no draft)	33.2	1.00x
Ngram (len=3)	34.6	1.04x
EAGLE-3 (len=3)	15.6	0.47x
EAGLE-3 (len=5)	11.8	0.36x

Note: EAGLE-3 speedup requires GPU acceleration. On CPU-only, the dual-model overhead outweighs speculation benefits for small models (0.6B). Speedup expected with GPU offloading or larger target models.

Training Details

Target model: Qwen3-0.6B (1024 dim, 3072 FFN, 28 layers)
Draft architecture: EAGLE-3 with 1 Eagle3Block (1024 hidden, 8 heads, 4 KV heads)
Draft vocabulary: 60K tokens (Fortran-optimized selection)
Training data: 500 ShareGPT conversations
Loss: Cross-entropy with cosine annealing + warmup
Hardware: CPU-only (no CUDA), PyTorch 2.13.0
Fortran integration: Acceptance sampling, logit correction, and rejection sampling kernels

Architecture

EAGLE-3 extracts hidden states from target model layers [2, 14, 25], projects them through a feature compressor, and feeds them into a lightweight transformer block for draft prediction.

ichsanlook/qwen3-eagle

Qwen3-0.6B EAGLE-3 Draft Model

Files

Usage with llama.cpp

Usage with Ollama

Benchmark (CPU-only, 16GB RAM)

Training Details

Architecture

ichsanlook/
qwen3-eagle