Two compact GGUF files of Qwen3.8-27B (the native vision-language 27B dense model, Gated DeltaNet + Gated Attention hybrid layout, 262,144-token native context), converted from unsloth/Qwen3.8-27B-NVFP4 — without the MTP speculative head.
These are the budget members of the esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF family: they share the same byte-identical 448-tensor NVFP4 backbone (all 64 layers' attention + MLP, 13.69 GB) as the compact MTP tiers, but the MTP draft block has been stripped and the extra tensors pinned to the smallest types. No drafter, no speculative decoding — just the model at the lowest possible footprint.
The goal here is to let 16 GB VRAM users benefit from native NVFP4. These are the smallest possible ways to serve this model on a single 16 GB Blackwell card. There are, of course, accuracy trade-offs vs the larger tiers, plus the lack of MTP — like every compact tier of the family, the attention path is re-quantized to NVFP4 (a second quantization step from the source's F8), and here the head tensors are pinned to the smallest types. You can also fit Qwen3.8-27B in a 16GB VRAM GPU through non-NVFP4 variants instead - e.g. those from Unsloth's own original GGUF repository (unsloth/Qwen3.8-27B-GGUF).
STARVED — the absolute floor: Q2_K lm_head andQ2_K token embedding. Smallest possible variant of the family.
Follow along & support
I post updates on new conversions, benchmarks, and what I'm working on over on Ko-fi. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, a coffee is always welcome. I do this on consumer hardware and like seeing how far it goes. More is on the way.
Both contain 1,187 tensors: the 448-tensor NVFP4 backbone (byte-identical across the whole family, verified per-tensor SHA-256) + 737 F32 norms/scales/gates + the two head tensors. qwen35.nextn_predict_layers=0 and qwen35.block_count=64 (the MTP block is gone), so they load in any llama.cpp build without the draft-mtp path.
Why these exist
The MTP family is built around a byte-identical NVFP4 backbone; the MTP draft head costs VRAM and compute on every pass. If you don't use speculative decoding, the stripped versions free that head entirely and shave the extra tensors down to the cheapest types — the smallest possible way to serve this model on a single 16 GB Blackwell card.
Vision works. The model is a native VLM (images and video). Pair either GGUF with the mmproj-BF16.gguf from the MTP repo (byte-identical to unsloth/Qwen3.8-27B-GGUF's projector) via --mmproj. If you need the absolute minimum VRAM footprint on a single 16 GB card, you can also load the model without the projector (no --mmproj, i.e. text-only) — that frees the projector's ~1 GB and is another lever to fit bigger context.
Benchmarks (32k context, q4_0 KV, one fresh pod per tier)
Hardware: a single RTX 5070 Ti (16 GB Blackwell). Payload: a 28k-token source document, max_tokens 2000. Single-run, indicative only. On reading the numbers: prefill t/s is an average over the run and degrades as context grows, so these figures are not comparable to numbers from different context sizes, hardware, or harnesses (e.g. the 180k-context observations or the RTX 5090 community benchmark in the MTP repo). Compare only within the same context and setup.
Tier
Prefill t/s
Decode t/s
req s
content distinct_ratio
BUDGET (Q3_K / Q2_K)
2287.3
27.05
126.5
0.9877
STARVED (Q2_K / Q2_K)
2289.2
27.37
98.7
1.0
Both are near-identical on prefill (~2288 t/s) and decode (~27.2 t/s); STARVED is ~1% faster on both.
Single-16GB VRAM probes (RTX 5070 Ti): BUDGET peaks 15.9 GiB, STARVED 15.7 GiB — both fit one 16 GB Blackwell card.
The full MTP family (ORIG + VERY-LOW / LOW / MEDIUM / HIGH / VERY-HIGH / HIGHEST, all with the speculative head baked in) lives in esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF.
License
Apache-2.0, identical to every upstream artifact. The base model license governs; GGUF conversion and quantization are transformations, not new training. When redistributing, please retain attribution to Qwen (Alibaba) and Unsloth as above.
"Qwen" is a trademark of Alibaba. Trademarks are used here only to identify upstream models; this repository is not affiliated with, sponsored by, or endorsed by Alibaba or Unsloth.
Note on this card
This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.
