This repository contains imatrix-aware, layer-aware Mixture-of-Quantization (MoQ) GGUF files for the original Qwen/Qwen3.8-27B model. These files preserve the model's MTP tensors and are quantizations of the base Qwen model. In terms of overall quality, MoQ is comparable to Unsloth's Dynamic Quantization 3.0; however, our quantization series offers a more granular range of options (4.0 bpw – 4.75 bpw) than the Unsloth team's offerings, while maintaining a slight performance edge over Dynamic Quantization 3.0 within this range.
The complete series was evaluated locally alongside the Unsloth Dynamic GGUF series under identical conditions. Lower is better in all three charts.
p999 KLD comparison
Mean KLD comparison
WikiText-2 PPL comparison
The interactive comparison report supports pan, zoom, view reset, and per-series visibility controls. The companion CSV contains all measured values and tensor-composition summaries.
Comparison Summary
All 17 measured GGUF files contain 866 tensors and 27,320,697,856 parameters. They were evaluated against the same BF16 reference logits on WikiText-2 with a fixed context length of 512.
At the exact size of each of the 9 MoQ files, linearly interpolating the Unsloth curve favors this MoQ series on:
PPL: 8 of 9 points
p999 KLD: 7 of 9 points
Mean KLD: 2 of 9 points
The results show the intended tradeoff clearly. The layer-aware MoQ recipes are especially effective on PPL and tail divergence, while the Unsloth Dynamic recipes remain very strong on Mean KLD, particularly at low and middle file sizes. The full table is included below so that users can choose based on the metric that matters for their workload.
Full Quality Results
Actual BPW is computed from the complete GGUF file size, including metadata and alignment. GB is decimal. PPL and all KLD values are lower-is-better; same top-p is higher-is-better.
Series
Recipe
Actual BPW
GB
PPL
Mean KLD
p999 KLD
p99 KLD
RMS delta-p
Same top-p
Jianqiao1 MoQ
3.2
3.0956
10.572
7.408463
0.111637
3.348125
1.084846
9.634%
85.679%
Jianqiao1 MoQ
3.6
3.4821
11.892
7.221080
0.072244
2.351531
0.717370
7.682%
88.722%
Jianqiao1 MoQ
3.8
3.7232
12.715
7.122706
0.048190
1.624749
0.437946
6.077%
90.109%
Jianqiao1 MoQ
4.1
4.1395
14.137
7.068801
0.031460
1.056327
0.285069
4.915%
91.872%
Jianqiao1 MoQ
4.3
4.3600
14.890
7.045628
0.024500
0.805452
0.216612
4.332%
92.648%
Jianqiao1 MoQ
4.6
4.4302
15.129
7.014436
0.020171
0.758995
0.200817
3.936%
93.809%
Jianqiao1 MoQ
4.8
4.6961
16.038
7.008832
0.017282
0.650152
0.169743
3.584%
94.339%
Jianqiao1 MoQ
4.9
4.8055
16.411
7.021034
0.016342
0.651529
0.162814
3.530%
94.423%
Jianqiao1 MoQ
5.1
5.0772
17.339
7.020549
0.012857
0.487299
0.123491
3.134%
94.968%
Unsloth
UD-IQ2_XXS
2.6383
9.010
7.957195
0.163689
4.714475
1.739405
11.822%
82.389%
Unsloth
UD-IQ2_M
3.0219
10.320
7.507406
0.103564
3.590751
1.076759
9.332%
85.496%
Unsloth
UD-Q2_K_XL
3.1263
10.676
7.403969
0.091579
3.107871
0.925859
8.708%
86.160%
Unsloth
UD-IQ3_XXS
3.4885
11.914
7.246240
0.054517
2.259040
0.627471
6.766%
90.359%
Unsloth
UD-Q3_K_XL
3.9358
13.441
7.111811
0.031355
1.248320
0.353498
5.129%
92.396%
Unsloth
UD-Q4_K_XL
5.2483
17.923
6.976017
0.008640
0.422880
0.090719
2.614%
96.073%
Unsloth
UD-Q5_K_XL
5.9203
20.218
6.967300
0.004526
0.200576
0.040594
1.849%
97.153%
Unsloth
UD-Q6_K_XL
7.5911
25.924
6.953912
0.001331
0.062800
0.011536
1.107%
98.493%
The Unsloth rows are local comparison measurements only. Unsloth GGUF files are not redistributed in this repository.
Available Models
The recipe number is a series label. Actual BPW below is calculated from the complete file and is the value to use for exact size comparisons.
File
Actual BPW
Size GB
Size GiB
Qwen3.8-27B-MTP-MoQ-3.2.gguf
3.0956
10.572
9.846
Qwen3.8-27B-MTP-MoQ-3.6.gguf
3.4821
11.892
11.075
Qwen3.8-27B-MTP-MoQ-3.8.gguf
3.7232
12.715
11.842
Qwen3.8-27B-MTP-MoQ-4.1.gguf
4.1395
14.137
13.166
Qwen3.8-27B-MTP-MoQ-4.3.gguf
4.3600
14.890
13.867
Qwen3.8-27B-MTP-MoQ-4.6.gguf
4.4302
15.129
14.090
Qwen3.8-27B-MTP-MoQ-4.8.gguf
4.6961
16.038
14.936
Qwen3.8-27B-MTP-MoQ-4.9.gguf
4.8055
16.411
15.284
Qwen3.8-27B-MTP-MoQ-5.1.gguf
5.0772
17.339
16.148
The repository also includes the locally generated calibration imatrix and a BF16 multimodal projector.
Quantization Approach
This series combines the advantages of the MoQ (Mixed-precision Quantization) scheme at the layer level with tensor-specific strategies derived from Unsloth Dynamic 3.0's low-bit GGUF implementations. Since Qwen3.8 and Qwen3.6 share a matching tensor-level architecture, we were able to transfer and fine-tune layer-specific quantization strategies without being forced to apply a uniform quantization type across all tensor families.
The specific process is as follows:
Generate a calibration imatrix specifically for Qwen3.8 using a calibration corpus.
Transfer the layer-wise MoQ allocation scheme from Qwen3.6 to the corresponding tensors in Qwen3.8.
Compare each tensor family and layer against the original weights.
Retain MoQ layer-wise precision adjustments (whether increasing or decreasing precision) if they improve perplexity (PPL) or KLD tail performance.
While maintaining the layer-wise structure, optimize average KLD performance for the low-BPW (bits per weight) schemes of versions 3.2 and 3.6 by referencing Unsloth's low-bit quantization strategies.
This is not a single, global quantization preset; different precision settings can be applied to specific tensor families and layers based on their measured or inferred sensitivity.
MTP Precision
Compared to the MTP layer in Qwen 3.6, the MTP layer in Qwen 3.8 employs lower-bit quantization; this is because our tests revealed that the prediction performance of Qwen 3.8's MTP layer—even at Q8 precision—falls short of that of Qwen 3.6, rendering the maintenance of high precision for the MTP layer less meaningful.
Evaluation Conditions
The comparison used llama.cpp build 10276, commit 6ea215d17, with CUDA on an NVIDIA GeForce RTX 5090. WikiText-2 wiki.test.raw was evaluated against BF16 logits from the original Qwen3.8-27B GGUF. Context length was fixed at 512; logical batch was 2048, ubatch was 8192, GPU layers were selected automatically with fit enabled, op offload and flash attention were enabled, and 16 CPU threads were used. GPU evaluations were run strictly one at a time.
The BF16 reference PPL reported by the evaluator was 6.950282.
Usage
Use a recent llama.cpp build with Qwen3.8 and MTP support. The files can be used for ordinary generation or MTP speculative decoding. Choose a file according to available memory and the quality curves above; the nominal recipe label is useful for navigating the series, while the actual file BPW and GB columns provide precise memory-planning values.
License and Acknowledgements
Released under the Apache License 2.0, following the base model license metadata.
Thanks to:
the Qwen team for Qwen3.8-27B and its MTP architecture;
the llama.cpp project and contributors for GGUF quantization, MTP support, and evaluation tooling;
