DFlash is a draft/speculative decoding model. It proposes tokens ahead of the main model, and the main model verifies them. When the draft tokens are accepted, generation can become faster.

In llama.cpp, this is used through:

bash

-md /path/to/dflash.gguf
--spec-type draft-dflash
--spec-draft-n-max 15

Download

Install the Hugging Face CLI if needed:

pip install -U huggingface_hub

Download this repository:

bash

hf download pythoneer/qwen36-35b-a3b-dflash-llamacpp \
  --local-dir ./qwen36-35b-a3b-dflash-llamacpp

Files

Expected GGUF file:

qwen36-35b-a3b-dflash-Q8_0.gguf

Basic llama.cpp usage

You need both:

The main/target model GGUF:

Qwen3.6-35B-A3B.gguf

This DFlash draft GGUF:

qwen36-35b-a3b-dflash-Q8_0.gguf

Run:

bash

./llama-server \
  -m /path/to/Qwen3.6-35B-A3B.gguf \
  -md ./qwen36-35b-a3b-dflash-llamacpp/qwen36-35b-a3b-dflash-Q8_0.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl 99 \
  -ngld 99 \
  -fa on \
  --jinja \
  -c 10000 \
  --host 0.0.0.0 \
  --port 8080

ROCm example

bash

./llama-server \
  -m /path/to/Qwen3.6-35B-A3B.gguf \
  -md /path/to/qwen36-35b-a3b-dflash-Q8_0.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl 99 \
  -ngld 99 \
  --device rocm0 \
  --spec-draft-device rocm0 \
  -fa on \
  --jinja \
  -c 10000 \
  -b 1024 \
  -ub 1024 \
  -np 1 \
  --host 0.0.0.0 \
  --port 8080

Debug/testing command

For first testing, use deterministic sampling:

bash

./llama-server \
  -m /path/to/Qwen3.6-35B-A3B.gguf \
  -md /path/to/qwen36-35b-a3b-dflash-Q8_0.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 8 \
  -ngl 99 \
  -ngld 99 \
  --jinja \
  -fa on \
  --temp 0 \
  --top-k 1 \
  -c 4096 \
  --host 0.0.0.0 \
  --port 8080

After confirming it works, increase:

bash

--spec-draft-n-max 15
-c 10000

and restore your preferred sampler settings.

Important notes

If you see this log during model loading or memory fitting:

dflash requires ctx_other to be set

it can be normal during memory measurement. The important test is whether the server loads and completes a request.

Do not run:

./llama-server -m qwen36-35b-a3b-dflash-Q8_0.gguf

This draft model needs the target model context and must be loaded with:

-md qwen36-35b-a3b-dflash-Q8_0.gguf

Device notes

For best stability, keep the main model and DFlash draft model on the same backend/device first.

Good:

bash

--device rocm0
--spec-draft-device rocm0

Good:

bash

--device cuda0
--spec-draft-device cuda0

Avoid mixing backends such as ROCm for the target model and Vulkan for the draft model until you verify it works on your build.

If you use tensor overrides like:

--override-tensor 'output.weight=CPU'

then also try:

--no-mmap

Conversion reference

This GGUF was made using llama.cpp conversion with the target model tokenizer/config:

bash

python convert_hf_to_gguf.py \
  /path/to/Qwen3.6-35B-A3B-DFlash \
  --target-model-dir /path/to/Qwen3.6-35B-A3B \
  --outfile qwen36-35b-a3b-dflash-Q8_0.gguf

The --target-model-dir directory needs the tokenizer/config files from the target model. It does not necessarily need the full target model weights.

Credits

Target model: Qwen/Qwen3.6-35B-A3B
Runtime: llama.cpp
Speculative mode: draft-dflash

Disclaimer

This repository only contains a converted draft model for llama.cpp use.

Please follow the license and usage terms of the original model.