Q4_0_ROCMFP4_STRIX_LEAN GGUF quantization of Qwen/Qwen3.6-35B-A3B (Qwen3.5-MoE architecture, ~3B active, multimodal), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

⚠️ These files use ROCmFP4 tensor types (ggml type 106) — stock llama.cpp will reject them (invalid ggml type). Use either:

kyuz0/amd-strix-halo-toolboxes rocm-7.2.4-rocmfp4 image (public, maintained), or

the Hal0ai/Hal0_ROCmFPX llama.cpp fork.

First published STRIX_LEAN of this model as far as I can tell (a ROCMFP4_FAST and an MTP-focused ROCmFP4 exist elsewhere — the STRIX_LEAN recipe trades a hair of file size for the Strix attn-K/V quality layout + Q5_K token embeddings).

Files

File	Quant	BPW	Size	Notes
`Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf`	Q4_0_ROCMFP4_STRIX_LEAN	4.29	19.0 GB	Fits a 64 GB carve-out with long context to spare
`mmproj-BF16.gguf`	BF16	—	0.90 GB	Vision projector — load alongside the quant
`imatrix_unsloth.gguf_file`	—	—	184 MB	Importance matrix used for calibration (from unsloth)

Measured performance

AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X), ROCm backend, kyuz0 rocm-7.2.4-rocmfp4 toolbox, llama-bench -fa 1 --mmap 0:

test	tok/s
pp512	1102
tg128	63.1

For reference, the same model as UD-Q4_K_XL runs 48 tok/s on identical hardware — the ROCmFP4 kernels are worth ~+31% decode on Strix Halo.

How it was made

BF16 GGUF source and imatrix from unsloth/Qwen3.6-35B-A3B-MTP-GGUF:

bash

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf \
  Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

(nextn/MTP metadata from the source survives conversion, though speculative decoding showed no speedup on Strix Halo in my testing — plain decode is recommended.)

Serving

bash

llama-server -m Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BF16.gguf -ngl 999 -fa on --jinja -c 65536

Companion repos

Credits

Base model: Qwen — Qwen3.6-35B-A3B (Apache-2.0)
BF16 GGUF conversion + imatrix: unsloth
ROCmFP4 quant formats: Hal0ai/Hal0_ROCmFPX; maintained builds: kyuz0

gsrunion/Qwen3.6-35B-A3B-ROCmFP4-STRIX_LEAN-GGUF

Qwen3.6-35B-A3B — ROCmFP4 STRIX_LEAN quant for AMD Strix Halo