Qwen3.8-27B MQ V2 ladder for hipfire
Text-only Qwen3.8-27B artifacts for
hipfire, the Rust-native LLM inference engine for AMD GPUs.
This repository contains the admitted MQ3V2 through MQ6V2 product ladder and matching DFlash drafters. MQ4V2 supersedes MQ4V1 as hipfire's canonical mq4 product. Artifacts remain self-describing by HFQM quant-type ID; old MQ4V1 files are still readable but are not the current product.
Product filenames
Each bit-width has exactly three tiers:
mqN-xt: embeddings protected; lm_head and recurrent outputs remain at the base codec.mqN: embeddings and lm_head protected.mqN-pro: recurrent/SSM outputs also lifted. MQ3 Pro uses ssm_out:mq6v2 to stay inside the 3.x model-bpw class.
| Product | File | Model bpw | Size (GB) | WT2 KLD |
|---|
| mq3-xt | qwen3.8-27b.mq3-xt | 3.503 | 11.778 | 0.248348 |
| mq3 | qwen3.8-27b.mq3 | 3.753 | 12.619 | 0.153658 |
| mq3-pro | qwen3.8-27b.mq3-pro | 3.922 | 13.184 | 0.130314 |
| mq4-xt | qwen3.8-27b.mq4-xt | 4.456 | 14.980 | 0.057449 |
| mq4 | qwen3.8-27b.mq4 | 4.659 | 15.663 | 0.039033 |
| mq4-pro | qwen3.8-27b.mq4-pro | 4.897 | 16.464 | 0.032495 |
| mq5-xt | qwen3.8-27b.mq5-xt | 5.408 | 18.183 | 0.015028 |
| mq5 | qwen3.8-27b.mq5 | 5.564 | 18.706 | 0.010255 |
| mq5-pro | qwen3.8-27b.mq5-pro | 5.746 | 19.319 | 0.009006 |
| mq6-xt | qwen3.8-27b.mq6-xt | 6.361 | 21.386 | 0.004389 |
| mq6 | qwen3.8-27b.mq6 | 6.469 | 21.750 | 0.002771 |
| mq6-pro | qwen3.8-27b.mq6-pro | 6.596 | 22.174 | 0.002208 |
MQ2V2 was fully measured but is deliberately not published as a product: all three tiers produced catastrophic WT2 KLD around 12–13 and degraded/repetitive text.
DFlash drafters
| Draft | File | Size (GB) | Role |
|---|
| MQ3V2 | qwen38-27b-dflash-mq3.hfq | 0.985 | Optional same-bit controller |
| MQ4V2 | qwen38-27b-dflash-mq4.hfq | 1.210 | Recommended controller |
| MQ5V2 | qwen38-27b-dflash-mq5.hfq | 1.434 | Optional same-bit controller |
| MQ6V2 | qwen38-27b-dflash-mq6.hfq | 1.659 | Optional same-bit controller |
Sizes are decimal GB, matching Hugging Face.
The MQ4V2 control drafter won the matched decode screen for every admitted MQ3/MQ5/MQ6 target tier and is therefore the recommended default controller. The same-bit drafts remain available for explicit size/codec experiments.
DFlash is opt-in:
1hipfire pull qwen3.8:27b
2hipfire pull qwen3.8:27b-draft-mq4
3hipfire run --spec dflash \
4 --model-draft ~/.hipfire/models/qwen38-27b-dflash-mq4.hfq \
5 qwen3.8:27b "Design a streaming Rust file deduplicator."
Usage
1# Canonical MQ4V2 base tier
2hipfire pull qwen3.8:27b
3hipfire run qwen3.8:27b "Design a streaming Rust file deduplicator."
4
5# Explicit ladder cells
6hipfire pull qwen3.8:27b-mq3-xt
7hipfire pull qwen3.8:27b-mq4-pro
8hipfire pull qwen3.8:27b-mq6
Runtime defaults
Every non-draft Qwen3.8 ladder tag receives the same registry tag policy:
| Setting | Value |
|---|
| KV cache | Q8 |
| KV backend | VMM |
| Native context | 262,144 tokens |
| Maximum output | 81,920 tokens |
| Temperature | 1.0 |
| Top-p | 0.95 |
| Top-k | 20 |
| Min-p | 0.0 |
| Presence penalty | 0.0 |
| Repeat penalty | 1.0 |
| Reasoning effort | xhigh |
| Thinking budget | uncapped |
Quality and performance evidence
Teacher KLD
The WT2 KLD values in the product table use 24 scored chunks against the
pinned WT2 teacher with Q8 KV and prefill scoring. They measure quantization
quality rather than runtime speed. A second 24-chunk validation-selection
fixture (v6sel) independently preserved the same tier ordering. MQ2V2 was
rejected before publication because its WT2 KLD was 12.44–13.24.
HumanEval pass@1
All 36 product × decode-mode arms ran the full 164-task HumanEval suite on
gfx1201: one stateless native-serve request per task, greedy sampling,
thinking off, Q8/VMM KV, and an 81,920-token output ceiling. DFlash rows use
the recommended MQ4V2 controller; MTP rows use the matching product sidecar.
HE/2 + HE/3 is the explicit smoke pair used throughout ladder bring-up.
| Product | AR pass@1 | MTP pass@1 | DFlash pass@1 | HE/2 + HE/3 |
|---|
| mq3-xt | 94.51% | 95.12% | 93.90% | pass |
| mq3 | 96.95% | 96.95% | 95.73% | pass |
| mq3-pro | 96.95% | 97.56% | 96.95% | pass |
| mq4-xt | 95.12% | 95.73% | 95.73% | pass |
| mq4 | 97.56% | 97.56% | 96.95% | pass |
| mq4-pro | 97.56% | 97.56% | 97.56% | pass |
| mq5-xt | 98.78% | 98.17% | 98.78% | pass |
| mq5 | 97.56% | 97.56% | 98.78% | pass |
| mq5-pro | 98.17% | 98.17% | 98.17% | pass |
| mq6-xt | 98.78% | 98.78% | 98.78% | pass |
| mq6 | 98.17% | 98.78% | 98.78% | pass |
| mq6-pro | 98.17% | 98.78% | 98.17% | pass |
DFlash controller screen
This fresh-process gfx1201 screen compares each target with its same-bit draft
and with the MQ4V2 control draft. Decode is tok/s; $\tau$ is emitted tokens per
verification cycle. MQ4V2 won or tied every admitted target, so it is the
registry default while all four drafters remain downloadable.
| Target | Same-bit decode | MQ4 control decode | Same-bit $\tau$ | MQ4 control $\tau$ |
|---|
| mq3-xt | 237.4 | 264.8 | 11.70 | 13.11 |
| mq3 | 225.5 | 250.8 | 11.70 | 13.11 |
| mq3-pro | 221.2 | 248.1 | 11.70 | 13.11 |
| mq4-xt | 251.6 | 251.6 | 11.70 | 11.70 |
| mq4 | 263.3 | 263.3 | 13.11 | 13.11 |
| mq4-pro | 258.2 | 258.2 | 13.11 | 13.11 |
| mq5-xt | 221.3 | 225.2 | 13.11 | 13.11 |
| mq5 | 214.5 | 217.4 | 13.11 | 13.11 |
| mq5-pro | 213.2 | 217.0 | 13.11 | 13.11 |
| mq6-xt | 207.8 | 211.5 | 13.11 | 13.11 |
| mq6 | 202.8 | 206.6 | 13.11 | 13.11 |
| mq6-pro | 202.1 | 205.9 | 13.11 | 13.11 |
Native throughput
The full promoted product matrix on Radeon RX 7900 XTX (gfx1100) uses
fixed-shape hipfire bench --matrix with Q8/VMM KV, --spec off,
--backend noslots, --workload stateless, 5 measured runs, and 3 warmups.
TG128@128 generates 128 tokens from an initial context of 128 tokens. The
MQ4 rows are the 2026-08-24 post-multi-wave screen; MQ5 and MQ6 were refreshed
on 2026-08-26 at runtime commit 363e93d77.
| Product | PP512 tok/s | TG128@128 tok/s |
|---|
| mq4-xt | 883.00 | 47.74 |
| mq4 | 873.10 | 45.13 |
| mq4-pro | 831.80 | 43.58 |
| mq5-xt | 814.90 | 40.32 |
| mq5 | 803.30 | 38.95 |
| mq5-pro | 767.50 | 37.92 |
| mq6-xt | 803.90 | 27.34 |
| mq6 | 797.00 | 26.88 |
| mq6-pro | 759.60 | 27.75 |
These are fixture-bound medians, not performance floors. The complete
five-sample arrays, artifact hashes, and binary hashes are preserved in the
dated historical checkpoint.
An earlier base-tier screen at runtime commit 9b6a83ae8 used 3 measured runs,
5 warmups, and a different PP2048/single-token decode fixture:
| GPU | Product | PP512 tok/s | PP2048 tok/s | TG1@128 tok/s |
|---|
| Radeon RX 7900 XTX (gfx1100) | mq4 | 995.31 | 972.61 | 45.18 |
| Radeon RX 7900 XTX (gfx1100) | mq5 | 807.00 | 787.49 | 38.01 |
| Radeon RX 7900 XTX (gfx1100) | mq6 | 800.04 | 781.35 | 25.93 |
| Radeon 8060S (gfx1151) | mq3 | 218.61 | 213.02 | 14.86 |
| Radeon 8060S (gfx1151) | mq4 | 376.01 | 365.17 | 13.46 |
| Radeon 8060S (gfx1151) | mq5 | 164.83 | 160.80 | 11.60 |
| Radeon 8060S (gfx1151) | mq6 | 193.18 | 186.80 | 9.52 |
The gfx1100 MQ3V2 batched WMMA route remains deliberately unpromoted after a
device-loss incident in DFlash verification; AR uses the proven row-wise path.
On gfx1201, the full HumanEval AR runs above measured median prefill/decode
throughput of 514.35/37.3 tok/s (mq3), 623.5/33.1 (mq4), 470.2/28.1 (mq5),
and 582.0/25.1 (mq6). HumanEval prompt lengths vary, so those gfx1201 medians
are not fixed-shape PP512 measurements and are reported separately.
Wire format
MQ V2 uses an 8-byte dual-FP16 affine header per 256-weight group: one scale/zero pair for weights 0–127 and one for 128–255. Wire IDs are MQ4V2 qt44, MQ6V2 qt47, MQ5V2 qt48, and MQ3V2 qt49.
License and attribution
The upstream Qwen3.8 weights are Apache-2.0 licensed. These quantized artifacts were produced for hipfire using its MagnumQuant FWHT-rotated quantization family. See the upstream model card and
hipfire provenance.
