This model ships a multi-token-prediction (nextn) head, and every quant here keeps that head unquantized at BF16 while the other 64 layers are quantized normally:

qwen35.block_count          = 65      # 64 transformer layers + 1 MTP layer
qwen35.nextn_predict_layers = 1
blk.64.*                    = bf16    # 424.7M params, left alone

Files

File	Bits	Size	Notes
`BF16/…-BF16-*.gguf`	16	~55 GB	Full precision, split into shards. Convert your own quants from this.
`…-Q8_0.gguf`	8	~29 GB	Near-lossless.
`…-Q6_K.gguf`	6	~23 GB	Very close to Q8_0 at meaningfully smaller size.
`…-Q5_K_M.gguf`	5	~20 GB	Strong quality/size balance.
`…-Q5_K_S.gguf`	5	~19 GB
`…-Q4_K_M.gguf`	4	~17 GB	Recommended default for most users.
`…-Q4_K_S.gguf`	4	~16 GB	Slightly smaller than Q4_K_M.
`…-IQ4_NL.gguf`	4	~16 GB	Non-linear 4-bit.
`…-IQ4_XS.gguf`	4	~16 GB	Smallest of the 4-bit family.
`…-Q3_K_L.gguf`	3	~15 GB
`…-Q3_K_M.gguf`	3	~14 GB
`…-Q3_K_S.gguf`	3	~13 GB
`…-Q2_K.gguf`	2	~11 GB	Noticeable quality loss.
`mmproj-F32.gguf`	32	1.8 GB	Vision projector, full precision.
`mmproj-BF16.gguf`	16	0.9 GB	Vision projector, bfloat16.
`mmproj-F16.gguf`	16	0.9 GB	Vision projector, float16. Fine for nearly everyone.

Usage

Text:

llama-cli -m Qwen3.8-27B-Fable-Distill-Q4_K_M.gguf -c 8192 -p "Hello"

Vision — pass the projector alongside the model:

bash

llama-mtmd-cli -m Qwen3.8-27B-Fable-Distill-Q4_K_M.gguf \
               --mmproj mmproj-F16.gguf \
               --image photo.jpg -p "Describe this image."

Server:

llama-server -m Qwen3.8-27B-Fable-Distill-Q4_K_M.gguf --mmproj mmproj-F16.gguf

Server + MTP:

llama-server -m Qwen3.8-27B-Fable-Distill-Q4_K_M.gguf --mmproj mmproj-F16.gguf --spec-type draft-mtp --spec-draft-n-max 3

Notes

The model is multimodal (image-text-to-text). Without an mmproj-*.gguf you get a text-only model. Three precisions are provided; F16 is the usual choice, BF16 matches the source weights' dtype, and F32 is there if you want the projector left entirely unquantized.
Qwen3.5-family chat template with thinking support: it accepts enable_thinking and a reasoning_effort of low, medium or xhigh (the template's own default is xhigh, which thinks at length every turn).
Base model sampling recommendations: temperature 1.0, top_p 0.95, top_k 20.

The data for this model was easily formatted, validated, and masked using Teich

This qwen3_5 model was trained 2x faster with Unsloth and Huggingface's TRL library.

TeichAI/Qwen3.8-27B-Fable-Distill-GGUF

Qwen3.8-27B-Fable-Distill — GGUF

MTP head kept at BF16

Files

Usage

Notes

TeichAI/
Qwen3.8-27B-Fable-Distill-GGUF