First ROCmFP4/ROCmFPX quantization of Qwen/Qwen3.5-27B. Four quants, built from the Hub BF16 GGUF (no re-conversion), each individually verified on real hardware.

Built for AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified — with the ROCmFPX llama.cpp fork.

⚠️ Needs a ROCmFPX-capable llama.cpp build. These will not load in stock llama.cpp / Ollama / LM Studio.

Variants

file	ftype	size	`token_embd`	decode	correctness
`Qwen3.5-27B-Q4_0_ROCMFP4_COHERENT.gguf`	102	14.72 GiB	Q6_K	12.08 t/s	3/3
`Qwen3.5-27B-Q6_0_ROCMFPX_AGENT.gguf`	114	23.22 GiB	Q8_0	7.11 t/s	3/3
`Qwen3.5-27B-Q8_0_ROCMFPX.gguf`	111	25.92 GiB	Q8_0	7.17 t/s	3/3
`Qwen3.5-27B-Q8_0_ROCMFPX_AGENT.gguf`	115	26.28 GiB	Q8_0	7.22 t/s	3/3

mmproj-BF16.gguf is included — required for image input (-fa off).

Which to pick: the 4-bit (102) — fastest and smallest, same answers as the 8-bits.

⚠️ Q6_0_ROCMFPX_AGENT (114) is the weakest choice here: larger than the 4-bit and slower than the plain 8-bit. The AGENT recipe raises speculative-draft acceptance, and Qwen3.5-27B ships no drafter, so that benefit cannot apply. It is included for completeness.

Measured — not estimated

Hardware: Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified. -ngl 999 -c 4096 -fa on -fit off. 300 tokens, warm-up discarded, median of 3.

ftype	run 1 / 2 / 3	median	spread
102	12.09 12.04 12.08	12.08	1.004
114	7.06 7.11 7.11	7.11	1.007
111	7.11 7.17 7.17	7.17	1.008
115	7.13 7.22 7.24	7.22	1.015

No MTP/EAGLE drafter ships with this model, so these are the deployed numbers with nothing left on the table.

Speculative decoding

Qwen ships no official MTP/draft head for Qwen3.5-27B — verified against the base repo file list and config.json (unlike Qwen3.8-27B, which does). The decode figures above are therefore no-speculation numbers.

⭐ However, classic draft-model speculation should work, because every Qwen3.5 model shares the same vocabulary (248320) and architecture (qwen35):

bash

llama-server -m Qwen3.5-27B-Q4_0_ROCMFP4_COHERENT.gguf \
  --spec-type draft-simple --model-draft Qwen3.5-0.8B-Q4_0_ROCMFP4_COHERENT.gguf \
  --spec-draft-ngl 99 --spec-draft-n-max 3 -ngl 999 -c 4096 -fa on -fit off

⚠️ We have not measured this — the box was needed elsewhere before the test completed. It is offered as a documented, plausible route, not a benchmarked claim. If you try it, the levers that mattered on comparable models were --spec-draft-n-max (sweep 2–5) and using the smallest drafter that still accepts well.

Third-party heads exist (VirVen/Qwen3.5-27B-EAGLE3-v2, selimaktas/Flow-Drafter-Qwen3.5-27B) but are SGLang-specific or research artifacts and do not load in llama.cpp.

Head protection

Qwen3.5-27B does not tie embeddings — it has a real output.weight, and both it and token_embd are protected (Q6_K on the 4-bit, Q8_0 on the rest). Audited by exact tensor name. Note the smaller Qwen3.5 models do tie, so the same flags behave differently across this family — pass both.

  15804651168  Qwen3.5-27B-Q4_0_ROCMFP4_COHERENT.gguf
  24936068768  Qwen3.5-27B-Q6_0_ROCMFPX_AGENT.gguf
  27834807968  Qwen3.5-27B-Q8_0_ROCMFPX.gguf
  28214425248  Qwen3.5-27B-Q8_0_ROCMFPX_AGENT.gguf

Credits

Base model: Qwen/Qwen3.5-27B
Quantization stack: ROCmFPX

kingjones777/Qwen3.5-27B-ROCmFP4-GGUF

Qwen3.5-27B — ROCmFP4 / ROCmFPX GGUF

Variants

Measured — not estimated

Speculative decoding

Head protection

Credits

kingjones777/
Qwen3.5-27B-ROCmFP4-GGUF