Views
No views yet
Q6_K quality, ~30% faster prompt-processing on AMD Strix Halo (gfx1151)unsloth/Qwen3.6-27B-MTP-GGUF — the general-purpose base, with MTP speculative decoding + Qwen3-VL vision. Neutral imatrix calibration, so base behavior is preserved (not skewed toward code).| Quality | ≈ stock Q6_K (PPL within error) |
| Prompt processing | +29% vs Q6_K (short ctx) → +20% at 64k — measured |
| Decode | ~18 tok/s with MTP (~9 raw) |
| Vision | Qwen3-VL — bundled mmproj/ |
⚠️ Requires the ROCmFPX fork (buildmain— the FP* types are merged in) — custom AMD quant types (enum IDs 110–115), not upstream-stable. Won't load in stock llama.cpp / LM Studio / Ollama. HF's precision badge is wrong — pick the file by name.
| File suffix | Size | Best for |
|---|---|---|
…embF16-headQ6-Q6_0_ROCMFPX.gguf ★ | 23G | best overall — the flagship |
…embF16-Q8_0_ROCMFPX.gguf | 28G | maximum fidelity |
…embF16-Q4_0_ROCMFP4.gguf | 19G | fastest decode (4-bit) |
…embF16-Q3_0_ROCMFPX.gguf | 16G | smallest |
Qwen3.6-27B-STRIX-.1# build the fork once — main already has the ROCmFPX quant types
2git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
3JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
4
5# serve the flagship — MTP + vision
6HSA_OVERRIDE_GFX_VERSION=11.5.1 build-strix-rocmfp4/bin/llama-server \
7 -m Qwen3.6-27B-STRIX-embF16-headQ6-Q6_0_ROCMFPX.gguf \
8 -dev ROCm0 -ngl 999 -fa on -c 32768 \
9 --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 2 \
10 --jinja --mmproj mmproj/mmproj-F32.gguf --host 0.0.0.0 --port 8080--jinja so the model's own chat template emits them — no special parser flag needed.| File suffix | Preset | Enum | Size | Role |
|---|---|---|---|---|
embF16-headQ6-Q6_0_ROCMFPX.gguf | Q6_0_ROCMFPX | 110 | 23G | flagship — f16 emb + Q6_K head + imatrix |
embF16-Q8_0_ROCMFPX.gguf | Q8_0_ROCMFPX | 111 | 28G | highest fidelity |
embF16-Q4_0_ROCMFP4.gguf | Q4_0_ROCMFP4 | 100 | 19G | fastest decode (4-bit body) |
embF16-Q3_0_ROCMFPX.gguf | Q3_0_ROCMFPX | 112 | 16G | smallest |
Q4/Q8 but not Q6/Q3 — the latter aren't standard llama.cpp quant names; pick by filename.)| Metric | Value |
|---|---|
| Functional smoke | chat/coding/JSON/tool-call/coherency ✅ (5/5) |
PPL vs Q6_K | flagship 5.7030 vs stock Q6_K 5.6618 → Δ +0.73% (general slice, ctx 512) |
Q6_K| Context | Q6_K | flagship | Δ |
|---|---|---|---|
| pp512 | 188 | 242 | +29% |
| pp2048 | 191 | 245 | +28% |
| pp10k | 180 | 228 | +27% |
| pp16k | 172 | 217 | +26% |
| pp32k | 156 | 193 | +23% |
| pp64k | 133 | 159 | +20% |
llama-bench (both servers stopped, warm-up pass discarded). The gfx1151-tuned ROCmFPX kernels win the compute-bound prefill — the edge is largest at short context and narrows to +20% at 64k as O(n²) attention takes over. Decode is bandwidth-bound (≈ Q6_K raw), and MTP (--spec-type draft-mtp) ~doubles it in serving. Single-rep llama-bench; treat absolutes as ±a few %.