Qwen 3.6-27B Custom Imatrix Quants (IE-Series)
This repository provides highly optimized GGUF quantizations of Qwen 3.6-27B using advanced imatrix (Importance Matrix) calibration.
While the initial Q4_0-IE provided a solid baseline, the newly developed Q4H variant represents a significant jump in quality, outperforming standard Q4_K_M and Q4_1 quants in both perplexity and stability (KLD P99).
Methodology: The ActivEnergy "IE" Calibration
These quants were generated using a custom-curated calibration dataset. I began with the Bartowski Calibration v5 text as a foundation and integrated modifications designed to better capture high-precision logic and nuanced language patterns.
- Tool:
llama.cpp / llama-quantize
- Calibration: Hybrid IE-Custom + some of Bartowski v5 calibration text
- Primary Objective: Minimize KLD (Kullback–Leibler Divergence) outliers, particularly in the P99 range, to ensure model stability during long-form generation.
Performance & Compatibility
- Speed: Tested on RTX 5090, achieving approximately 60 TPS.
- Agents: Fully compatible with multi-agent frameworks like OpenClaw.
Evaluation Report: ActivEnergy Quantizer
The following metrics were derived using the Qwen 3.6-27B architecture as the base. The IE-Q4H variant is the current top recommendation for users seeking the best balance between size and logical fidelity.
Comparison Table
| Rank | Variant | File Size | Mean PPL | KLD Mean | KLD Median | KLD P99 |
|---|
| -- | F16 Baseline | 50.1 GB | 2.0878 | N/A | N/A | N/A |
| 1 | IE-Qwen36-27b-Q4H | 16.9 GB | 2.0844 | 0.010602 | 0.002350 | 0.084462 |
| 2 | Unsloth-UD-Q4_K_XL | 16.4 GB | 2.0716 | 0.008079 | 0.002192 | 0.146924 |
| 3 | IE-IQ4_NL | 14.7 GB | 2.1352 | 0.019256 | 0.002533 | 0.175252 |
| 4 | IE-IQ4_XS | 14.0 GB | 2.1263 | 0.018374 | 0.002328 | 0.184353 |
| 5 | Bartowski-Q4_1 | 16.4 GB | 2.0844 | 0.012945 | 0.002636 | 0.219050 |
Happy Inference! Please leave a comment with your results or any testing data.
