Unsloth-Qwen3.6-27B-NVFP4-A.gguf: The NVFP4 group 1 layers are preserved. The FP8 group 0 linear-attention (attn_qkv, ssm_out) and tail-MLP (layers 56–63) are converted to NVFP4, while the remaining layers are kept at Q8. The MTP draft layer is preserved at BF16. This was generated from NVFP4-Q8 using a modified llama-quantize to allow re-quantizing Q8-quantized layers to NVFP4.
Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf: The NVFP4 group 1 layers are preserved, while the FP8 group 0 layers are converted to Q8. This was generated using a modified convert_hf_to_gguf.py to support Unsloth's differing groups.
Unsloth-Qwen3.6-27B-NVFP4-BF16.gguf: The NVFP4 group 1 layers are preserved, while the FP8 group 0 layers are upcast to BF16. This version is not published because it is redundant with NVFP4-Q8.

Quantizations provided

File	Quantization	Size
Unsloth-Qwen3.6-27B-NVFP4-A.gguf	Mixed Q8/NVFP4 group 0, NVFP4 group 1	20.1 GB
Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf	Q8 group 0, NVFP4 group 1	23.2 GB

Perplexity test

I tested perplexity using llama-perplexity and Salesforce's wikitext-2-raw-v1.

File	Ctx	PPL
Unsloth-Qwen3.6-27B-NVFP4-A.gguf	512	7.5789 ± 0.05260
Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf	512	7.3792 ± 0.05057
Unsloth-Qwen3.6-27B-NVFP4-BF16.gguf	512	7.3702 ± 0.05052

Evaluation

The following models were evaluated for a fair comparison of capability, size and speed.

Model	Quantization	Size	Reason
CodeFault/Nvidia-Qwen3.6-27B-MTP-GGUF	NVFP4-A¹	17.9 GB	Alternative NVFP4 quant.
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL	20.4 GB	Closest non-NVFP4 in size to NVFP4-A.
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL	26 GB	Closest non-NVFP4 in size to NVFP4-Q8.
unsloth/Qwen3.6-27B-NVFP4	NVFP4²	25.4 GB	Previous NVFP4 checkpoint from 2026-05-31.

1: My quantization of nvidia/Qwen3.6-27B-MTP.

2: unsloth/Qwen3.6-27B-NVFP4 does not provide a GGUF. I used llama.cpp's conversion which passes through Unsloth's NVFP4 tensors. This is Unsloth's previous version from 2026-05-31.

	CodeFault Unsloth NVFP4-A	CodeFault Unsloth NVFP4-Q8	CodeFault Nvidia NVFP4-A	Unsloth NVFP4 (2026-05-31)	Unsloth UD-Q5_K_XL	Unsloth UD-Q6_K_XL
Coding
HumanEval	0.8354 ± 0.0290	0.8293 ± 0.0295	0.8293 ± 0.0295	0.8110 ± 0.0307	0.8598 ± 0.0272	0.8537 ± 0.0277
HumanEval+	0.7622 ± 0.0333	0.7622 ± 0.0333	0.7683 ± 0.0330	0.7744 ± 0.0327	0.7866 ± 0.0321	0.7805 ± 0.0324
MBPP	0.7540 ± 0.0193	0.7720 ± 0.0188	0.7280 ± 0.0199	0.7420 ± 0.0196	0.7540 ± 0.0193	0.7540 ± 0.0193
MBPP+	0.8836 ± 0.0165	0.8915 ± 0.0160	0.8783 ± 0.0168	0.8995 ± 0.0155	0.8968 ± 0.0157	0.8836 ± 0.0165
Instruction
IFEval	0.8540 ± 0.0152	0.8577 ± 0.0150	0.8558 ± 0.0151	0.8447 ± 0.0156	0.8558 ± 0.0151	0.8447 ± 0.0156
Knowledge
ARC-Challenge	0.9735 ± 0.0047	0.9718 ± 0.0048	0.9710 ± 0.0049	0.9710 ± 0.0049	0.9684 ± 0.0051	0.9710 ± 0.0049
MMLU-Pro	0.8291 ± 0.0034	0.8288 ± 0.0034	0.8250 ± 0.0034	0.8293 ± 0.0033		0.8319 ± 0.0033
STEM & Reasoning
BIG-Bench Hard	0.9201 ± 0.0032	0.9174 ± 0.0032	0.9149 ± 0.0032	0.9131 ± 0.0033	0.9220 ± 0.0031	0.9214 ± 0.0031
GPQA Diamond flexible	0.7929 ± 0.0289	0.7929 ± 0.0289	0.7929 ± 0.0289	0.8182 ± 0.0275	0.8131 ± 0.0278	0.7778 ± 0.0296
GSM8K	0.9151 ± 0.0077	0.9136 ± 0.0077	0.9196 ± 0.0075	0.9098 ± 0.0079	0.9128 ± 0.0078	0.9158 ± 0.0076
Hendrycks Math	0.3698 ± 0.0066	0.3624 ± 0.0066	0.3438 ± 0.0065	0.3662 ± 0.0066	0.3778 ± 0.0067	0.3876 ± 0.0067

NOTICE: These evals are actively running.

These evaluations were run using lm_eval. The models were run in instruct (non-thinking) mode with the following parameters in llama-server (b9775):

ini

ctx-size = 32768
cache-type-k = q8_0
cache-type-v = q8_0
top-p = 0.8
top-k = 20
min-p = 0
presence-penalty = 1.5
spec_type = draft-mtp
spec_draft_n_max = 2
chat-template-kwargs = {"enable_thinking":false}

Benchmarks

Model	Quant	MTP n-max	Prompt Len	Output Len	Acceptance Rate	pp/s	tg/s
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A		267	6654		2205.4	70.6
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	1	267	7240	0.880	1880.7	91.8
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	2	267	5842	0.856, 0.712	1853.9	109.8
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	3	267	5711	0.852, 0.712, 0.596	1782.3	119.7
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	4	267	9709	0.841, 0.688, 0.556, 0.462	1738.4	122.7
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-Q8		267	6544		2077.4	63.0
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-Q8	1	267	6596	0.876	1742.8	84.3
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-Q8	2	267	7088	0.868, 0.724	1766.4	102.8
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-Q8	3	267	5903	0.831, 0.675, 0.543	1785.1	108.8
CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF	NVFP4-Q8	4	267	5785	0.837, 0.684, 0.563, 0.471	1771.2	116.3
CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A		267	8676		2225.1	76.6
CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	1	267	5958	0.879	1877.0	99.5
CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	2	267	6708	0.856, 0.701	1897.2	119.1
CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	3	267	6900	0.855, 0.699, 0.559	1950.4	128.7
CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF	NVFP4-A	4	267	5635	0.836, 0.681, 0.567, 0.483	1862.6	137.6
unsloth/Qwen3.6-27B-NVFP4	NVFP4³		267	7347		2056.5	57.6
unsloth/Qwen3.6-27B-NVFP4	NVFP4³	1	267	6826	0.843	1749.9	74.9
unsloth/Qwen3.6-27B-NVFP4	NVFP4³	2	267	8142	0.851, 0.685	1794.7	90.1
unsloth/Qwen3.6-27B-NVFP4	NVFP4³	3	267	7612	0.837, 0.671, 0.541	1787	96.4
unsloth/Qwen3.6-27B-NVFP4	NVFP4³	4	267	7621	0.817, 0.620, 0.485, 0.400	1772.4	96.5
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL		267	7866		1522.0	64.0
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL	1	267	6977	0.866	1296.4	99.0
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL	2	267	7418	0.850, 0.686	1383.8	115.0
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL	3	267	9204	0.850, 0.684, 0.565	1305.6	118.9
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q5_K_XL	4	267	7600	0.855, 0.677, 0.565, 0.472	1294.0	123.5
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL		267	7180		1257.3	53.7
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL	1	267	5868	0.876	1249.1	84.8
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL	2	267	6104	0.864, 0.701	1232.8	102.2
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL	3	267	5000	0.847, 0.688, 0.563	1228.7	109
unsloth/Qwen3.6-27B-MTP-GGUF	UD-Q6_K_XL	4	267	7060	0.852, 0.703, 0.577, 0.474	1052.5	116.8

3: This is Unsloth's previous version from 2026-05-31.

These benchmarks were run on an RTX 5090 (limited to 480 W) using llama-cli (b9775) with the CUDA driver and a prompt to generate an Ansible playbook.

Serving with llama.cpp

It has a max context size of 262,114. This can be served using:

bash

llama-server \
-hf CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF \
-hff Unsloth-Qwen3.6-27B-NVFP4-A.gguf \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--repeat-penalty 1.1 \
--spec-type draft-mtp \
--spec-draft-n-max 3