Qwen3.6-27B — IQ2_XXS (WebGPU, runs headed on 12 GB cards)
A 2.06-bit-class (IQ2_XXS) imatrix quant of Qwen3.6-27B built to fit the full
27B model headed in the browser (i.e. while the same GPU is driving your
desktop display) on 12 GB cards — validated on the RX 6700 XT and
applicable to the RTX 3060 12 GB.
File:Qwen3.6-27B-IQ2_XXS-mtp.gguf
Size on disk: 8.88 GB (8462 MiB, 2.60 BPW)
Effective GPU weights: ~8.17 GB in the Artifex WebGPU engine (the MTP /
next-token-prediction block is stripped on load — see notes)
Calibration: Unsloth imatrix (unsloth_calibration_Qwen3.6-27B.txt,
496 entries over 77 chunks)
Why this quant exists
The official Unsloth 2-bit builds are excellent but use the UD (dynamic)
recipe, which (a) lands at ~9.4–9.6 GB and (b) can promote sensitive tensors to
IQ types that some lightweight inference engines don't decode. This file is a
flat-ish stock-imatrix IQ2_XXS — smaller (8.88 GB), and it sticks to a quant
type set that the Artifex WebGPU engine decodes natively. That ~0.8 GB headroom
is the difference between loads headed and device-lost on a 12 GB card whose
compositor is also eating VRAM.
If you have VRAM to spare or run headless, prefer the official
unsloth/Qwen3.6-27B-MTP-GGUF
UD builds for slightly better quality.
Measured performance (greedy / deterministic)
Scenario
Card
Speed
Notes
Headless
RX 6700 XT (12 GB)
6.6 tok/s
clean EOS, coherent
Headed (display on same GPU)
RX 6700 XT (12 GB)
7.4 tok/s
no device-lost
llama.cpp parity (CPU)
—
2.3 tok/s
source coherence check
Decode is GPU-bound (the IQ2_XXS matmuls are ~82% of each forward pass), so
expect comparable numbers on the RTX 3060 12 GB.
Quality
This is a stock-imatrix IQ2_XXS, not Unsloth Dynamic, so it gives up a little
quality versus the UD-IQ2_XXS for ~0.8 GB less footprint. In greedy testing the
output is fully coherent and converges with the llama.cpp reference run — no
2-bit "word salad."
The 2-bit quant costs +2.31 PPL (~39%) versus the full-precision base — a
real but non-catastrophic loss. The model stays coherent; it is not degraded
into incoherence. (PPL at n_ctx=512 runs higher than the commonly-quoted
4096-context numbers; use the same settings if you compare.)
Running it
In the Artifex WebGPU engine (the intended target)
Get the engine from Artifex_Web and run it (see that repo's README for build/serve steps), then:
Open the WebGPU app in a WebGPU-capable browser (Chrome/Edge).
Enter this repo id in the model box: kavenmartinez/Qwen3.6-27B-IQ2_XXS-webgpu-GGUF
Select your GPU adapter, load, and chat.
AMD cards (RX 6700 XT) — required step: the engine's auto VRAM detection is
NVIDIA-only, so AMD cards fall back to a conservative default that's too small
for a 27B and the load will fail. Set a manual budget on the app's origin
before loading, via the browser devtools console:
localStorage.setItem('vramBudgetGB', '11.8'); // 12 GB card, ~1 GB left for the compositor
NVIDIA cards (RTX 3060 12 GB): auto-detection works; no override needed. If
you still hit an OOM headed, set vramBudgetGB to 11.0–11.5.
In llama.cpp / llama-server
The file keeps its MTP block (blk.64, q8_0), so llama.cpp users get the
multi-token-prediction speculative speedup for free:
On a 12 GB card you may need partial offload (-ngl < 99) depending on context
length and KV cache size.
Notes & internals
MTP block: the GGUF contains the next-token-prediction block at q8_0
(~590 MB). The Artifex WebGPU engine strips it on load (it uses an n-gram
drafter, not MTP), which is why effective GPU weights are ~8.17 GB rather than
the 8.88 GB disk size. Kept in the file so llama.cpp can use it.
Quant mix: predominantly IQ2_XXS, with embeddings/output and a handful of
sensitive tensors kept at higher precision (Q4_K / Q5_K / Q2_K), which is why
the BPW is 2.60 rather than a pure 2.06.
Tokenizer: include tokenizer.json and tokenizer_config.json in this
repo — the WebGPU engine loads them directly via AutoTokenizer.from_pretrained
for remote repos, so the model is self-contained.
Credits & license
Base model:Qwen3.6-27B by the Qwen team (Apache-2.0).
