Quantized from unsloth's BF16 GGUF (200.8 GiB, 5 shards, byte-verified against the Hub) — a
lossless source, not a requantization and not our own re-conversion.
⭐ --output-tensor-type q6_K is what actually protects the LM head — the COHERENT tier
alone still leaves output.weight at 4-bit. Audited in the finished file: output.weightq6_K,
token_embd.weightq6_K.
Measured
Ryzen AI MAX+ 395, gfx1151, ROCm. -ngl 999 -c 4096 -fa off -fit off --mmproj.
Median of 3, warm-up discarded, otherwise-idle box.
Decode
17.72 tok/s (17.69 / 17.72 / 17.74 — tight)
Load
48 s cold, 22 s warm
Correctness (max_tokens: 1024): 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo ·
days in 2024 ⇒ ✅ 366
Vision (-fa off + bundled BF16 mmproj):
image
result
256×256 solid red
✅ Red.
256×256 shapes
✅ red square · green circle · blue triangle · yellow border · black background
512×512
⛔ failed to encode image slice — server dies
⚠️ Use images at or near 256×256. 512×512 fails to encode on this build. The projector is
fine — 256×256 reads colour and shape and spatial layout correctly — but larger images hit an
encode-slice limit. Resize before sending.
No draft head. This model ships no MTP/EAGLE/nextn tensors
(common_speculative_init: no implementations specified), so there is no speculative-decoding
speedup to enable and no acceptance-length figure to quote.
What was NOT measured
No perplexity run, and no quality A/B against the BF16 source.
No long-context testing. · No tool-calling evaluation.
Vision was smoke-tested on synthetic images only — no vision benchmark.
Base model licence (Llama 4 Community License) inherited; all credit for the weights goes to Meta.
