This repository contains GGUF conversions and quantizations of RedHatAI/Qwen3-14B-speculator.eagle3 for use with llama.cpp EAGLE3 speculative decoding.

[!IMPORTANT] This is not a standalone chat model. It is an EAGLE3 draft/speculator model and must be used together with the matching target/verifier model.

Target model: Qwen/Qwen3-14B
Speculator source: RedHatAI/Qwen3-14B-speculator.eagle3
Runtime: llama.cpp with --spec-type draft-eagle3

Files

File	Type	Notes
`Qwen3-14B-speculator.eagle3-F16.gguf`	EAGLE3 speculator GGUF	Converted from the original RedHatAI safetensors checkpoint
`Qwen3-14B-speculator.eagle3-Q8_0.gguf`	Quantized EAGLE3 speculator GGUF	Quantized from the F16 GGUF
`Qwen3-14B-speculator.eagle3-Q4_K_M.gguf`	Quantized EAGLE3 speculator GGUF	Quantized from the F16 GGUF; may be faster for draft decoding

Usage with llama.cpp

Example:

bash

llama-server \
  -m Qwen3-14B-Q4_K_M.gguf \
  -md Qwen3-14B-speculator.eagle3-Q4_K_M.gguf \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 4 \
  --spec-draft-p-min 0.5 \
  -c 32768 \
  -ngl 99 \
  -fa on

Windows CMD example:

cmd

llama-server.exe ^
  -m Qwen3-14B-Q4_K_M.gguf ^
  -md Qwen3-14B-speculator.eagle3-Q4_K_M.gguf ^
  --spec-type draft-eagle3 ^
  --spec-draft-n-max 4 ^
  --spec-draft-p-min 0.5 ^
  -c 32768 ^
  -ngl 99 ^
  -fa on

PowerShell example:

powershell

.\llama-server.exe `
  -m "Qwen3-14B-Q4_K_M.gguf" `
  -md "Qwen3-14B-speculator.eagle3-Q4_K_M.gguf" `
  --spec-type draft-eagle3 `
  --spec-draft-n-max 4 `
  --spec-draft-p-min 0.5 `
  -c 32768 `
  -ngl 99 `
  -fa on

Important Notes

This GGUF file is only the draft/speculator model. You still need a compatible GGUF of the target model, such as a GGUF conversion or quantization of Qwen/Qwen3-14B.

Do not use this speculator with unrelated models such as Qwen3-8B, Qwen3-32B, Gemma, Llama, or Qwen3.6 models. EAGLE3 speculators are target-specific.

Even small differences in the target model, prompt format, quantization, or runtime settings may affect draft acceptance rate and overall speed.

Tested Configuration

Tested with:

Runtime: llama.cpp with EAGLE3 support
Target model: Qwen3-14B GGUF
Draft model: this EAGLE3 GGUF
Example settings:
- --spec-type draft-eagle3
- --spec-draft-n-max 4
- --spec-draft-p-min 0.5

Local benchmark observations may vary depending on GPU, quantization, context length, batch size, sampling settings, and prompt type.

Benchmark Notes

In local testing, EAGLE3 showed stronger gains on structured or predictable outputs such as:

code generation
JSON output
repeated pattern generation
summarization
code completion

It was less effective on some tasks such as translation, creative writing, and some open-ended prompts, where draft acceptance may be lower.

For this speculator, lower-bit draft quantization may perform better than F16 in some llama.cpp setups because the draft model becomes cheaper to run. In local testing, Q4_K_M was a strong candidate for practical use, while results may vary across hardware and runtime versions.

Conversion

Converted with llama.cpp convert_hf_to_gguf.py using the original speculator repository and the matching target model directory.

Example conversion command:

bash

python convert_hf_to_gguf.py \
  RedHatAI/Qwen3-14B-speculator.eagle3 \
  --outtype f16 \
  --target-model-dir Qwen/Qwen3-14B \
  --outfile Qwen3-14B-speculator.eagle3-F16.gguf

PowerShell example:

powershell

python .\convert_hf_to_gguf.py `
  "E:\OLLAMA_MODELS\Qwen3-14B-speculator.eagle3" `
  --outtype f16 `
  --target-model-dir "E:\OLLAMA_MODELS\Qwen3-14B" `
  --outfile "E:\OLLAMA_MODELS\Qwen3-14B-speculator.eagle3-F16.gguf"

Quantization

The F16 GGUF can be quantized with llama-quantize.

Q8_0 example:

bash

llama-quantize \
  Qwen3-14B-speculator.eagle3-F16.gguf \
  Qwen3-14B-speculator.eagle3-Q8_0.gguf \
  Q8_0

Q4_K_M example:

bash

llama-quantize \
  Qwen3-14B-speculator.eagle3-F16.gguf \
  Qwen3-14B-speculator.eagle3-Q4_K_M.gguf \
  Q4_K_M

PowerShell example:

powershell

.\llama-quantize.exe `
  "E:\OLLAMA_MODELS\Qwen3-14B-speculator.eagle3-F16.gguf" `
  "E:\OLLAMA_MODELS\Qwen3-14B-speculator.eagle3-Q4_K_M.gguf" `
  Q4_K_M

Credits

Original EAGLE3 speculator model by RedHatAI:

RedHatAI/Qwen3-14B-speculator.eagle3

Target model:

Qwen/Qwen3-14B

GGUF support and runtime:

ggml-org/llama.cpp

License

This repository is a converted GGUF version of the original speculator model. The original model license and usage terms apply. Please refer to the upstream repositories for full license details.

williamliao/Qwen3-14B-EAGLE3-Speculator-GGUF

Qwen3-14B EAGLE3 Speculator GGUF

Files

Usage with llama.cpp

Important Notes

Tested Configuration

Benchmark Notes

Conversion

Quantization

Credits

License

williamliao/
Qwen3-14B-EAGLE3-Speculator-GGUF