Views
No views yet
Q6_K quality, ~25% faster prompt-processing on AMD Strix Halo (gfx1151)Jackrong/Qwopus3.6-27B-v2-MTP-GGUF — the general-purpose 27B Qwopus, MTP speculative decoding + Qwen3-VL vision, agent/tool-use tuned.| Quality | ≈ stock Q6_K — PPL +0.28% (within error) |
| Prompt processing | +27% vs Q6_K (short ctx) → +16% at 64k |
| Decode | ~18 tok/s with MTP (~9 raw) |
| Vision | Qwen3-VL — bundled mmproj/ |
⚠️ Requires the ROCmFPX fork (buildmain— the FP* types are merged in) — custom AMD quant types (enum IDs 110–115), not upstream-stable. Won't load in stock llama.cpp / LM Studio / Ollama. HF's precision badge is wrong — pick the file by name.
| File suffix | Size | Best for |
|---|---|---|
…embF16-headQ6-Q6_0_ROCMFPX_AGENT.gguf ★ | 26 GB | best overall — the flagship |
…embF16-Q8_0_ROCMFPX.gguf | 29 GB | maximum fidelity |
…embF16-Q4_0_ROCMFP4.gguf | 19 GB | fastest decode (4-bit) |
…embF16-Q3_0_ROCMFPX.gguf | 17 GB | smallest |
_AGENT tiers + the full enum/bpw table are in the details below and the Files tab. All filenames prefixed Qwopus3.6-27B-v2-MTP-STRIX-.1# build the fork once — main already has the ROCmFPX quant types
2git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
3JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
4
5# serve the flagship — MTP + vision
6HSA_OVERRIDE_GFX_VERSION=11.5.1 build-strix-rocmfp4/bin/llama-server \
7 -m Qwopus3.6-27B-v2-MTP-STRIX-embF16-headQ6-Q6_0_ROCMFPX_AGENT.gguf \
8 -dev ROCm0 -ngl 999 -fa on -c 131072 \
9 --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 2 \
10 --jinja --mmproj mmproj/mmproj-F32.gguf --host 0.0.0.0 --port 8080qwen3_coder parser, or the model narrates instead of emitting structured calls.| File suffix | Preset | Enum | Size | Role |
|---|---|---|---|---|
embF16-headQ6-Q6_0_ROCMFPX_AGENT.gguf | Q6_0_ROCMFPX_AGENT | 114 | 26 GB | flagship — f16 emb + Q6_K head + imatrix |
embF16-Q8_0_ROCMFPX_AGENT.gguf | Q8_0_ROCMFPX_AGENT | 115 | 30 GB | highest-fidelity agent |
embF16-Q8_0_ROCMFPX.gguf | Q8_0_ROCMFPX | 111 | 29 GB | highest fidelity |
embF16-Q6_0_ROCMFPX.gguf | Q6_0_ROCMFPX | 110 | 24 GB | balanced |
embF16-Q3_0_ROCMFPX_AGENT.gguf | Q3_0_ROCMFPX_AGENT | 113 | 21 GB | smallest agent |
embF16-Q3_0_ROCMFPX.gguf | Q3_0_ROCMFPX | 112 | 17 GB | smallest |
embF16-Q4_0_ROCMFP4.gguf | Q4_0_ROCMFP4 | 100 | 19 GB | fastest decode (4-bit body) |
_AGENT presets keep attention/FFN routing at higher precision for tool-call/JSON coherence. (HF labels Q4/Q8 but not Q6/Q3 — the latter aren't standard llama.cpp quant names.)| Metric | Value |
|---|---|
| Functional smoke | chat/coding/JSON/tool-call/coherency ✅ (5/5) |
PPL vs Q6_K (code corpus) | flagship 2.962 vs Q6_K 2.954 → +0.28% (within ±0.04) |
Q6_K| Context | Q6_K | flagship | Δ |
|---|---|---|---|
| pp512 | 195 | 248 | +27% |
| pp2048 | 189 | 242 | +28% |
| pp10k | 179 | 222 | +24% |
| pp16k | 172 | 208 | +21% |
| pp32k | 157 | 186 | +19% |
| pp64k | 133 | 155 | +16% |
--spec-type draft-mtp) ~doubles it in serving. Q4_0_ROCMFP4 is the decode king (~13 tok/s raw). Single-rep llama-bench, ran warm — absolutes run a touch conservative vs the Coder sibling.