Qwen3.6-27B - GGUF (Bedrock / Tightrope / Gambit)
Three GGUF builds of
Qwen/Qwen3.6-27B, quantized starting from
unsloth/Qwen3.6-27B-GGUF's Q8_0 release.
What I did
Instead of picking one compression level and applying it across the whole model, I built a harness that quantizes one component at a time and measures how much it actually diverges from the source, using KL-divergence across general, code, math, and tool-calling prompts. That gave me a real, measured safe point for each component I tested, rather than a guess.
I also generated my own imatrix (calibration data) for this model rather than reusing the one bundled with the Q8_0 source, and used it for every quantization pass in this project.
What the testing showed
- Size doesn't predict compression tolerance. Components with similar size and similar architectural role sometimes had very different tolerance for compression, one held up fine where a similar one broke earlier. Not something you'd catch without testing directly.
- Divergence consistently started in a narrow range across the components I tested, roughly 3.5 to 3.9 bits per weight.
- Tool-calling was the first capability to show divergence in most components tested, ahead of general chat, code, or math.
These are findings from testing this specific model, against this specific Q8_0 source. I'm not claiming they generalize beyond it.
Measured KL-divergence (toolcalling, most sensitive category)
| Component | Safe floor | KLD at floor | Next level down | KLD at break |
|---|
| attn_qkv | IQ4_XS | 0.009 | Q3_K | 0.073 |
| ffn_down | IQ4_XS | 0.008 | Q3_K | 0.028 |
| ffn_up | IQ4_XS | 0.006 | Q3_K | 0.036 |
| output.weight | IQ4_XS | 0.006 | Q3_K | 0.034 |
| ssm_alpha | IQ2_XXS | 0.007 | — | survived entire tested ladder |
| token_embd | IQ3_S | 0.009 | IQ3_XXS | 0.016 |
These are per-component numbers from the isolated sweep, not from the combined final build (see Scope below).
The builds
| File | Size | True BPW |
|---|
| Bedrock | 13.26 GB | 4.23 |
| Tightrope | 12.53 GB | ~4.0 |
| Gambit | 10.94 GB | 3.49 |
- Bedrock: nothing compressed past what testing showed as clean. Closest to the source model, largest of the three.
- Tightrope: most components at their measured safe point, a few pushed one step further where the data supported it.
- Gambit: every tested component pushed one step past its safe point, on purpose, for the smallest footprint. Least margin of the three.
Testing
I ran all three builds by hand against real coding tasks, logic puzzles, and multi-step reasoning problems. Output quality was consistent with what I'd expect from the base model, occasional small, easily-caught mistakes (a typo, an off-by-one), nothing that looked like a real loss of reasoning ability.
This is manual, hands-on testing, not a formal benchmark suite. I'm not reporting a score.
Scope
- The per-component measurements were done in isolation; I haven't re-tested the fully combined model against the same measurement.
- A small number of components were left unchanged rather than individually tuned.
- All comparisons are against Unsloth's Q8_0 GGUF, not the original FP16/BF16 release. This means my baseline already carries whatever loss Unsloth's own quantization introduced, this project doesn't measure against the original weights directly.
Usage
Standard GGUF, works with llama.cpp, LM Studio, koboldcpp, and other GGUF-compatible runtimes.
1./build/bin/llama-cli \
2 -m Qwen3.6-27B-Bedrock-13.26GB.gguf \
3 -p "Explain quantum computing in simple terms." \
4 -n 256 \
5 --temp 0.7 --top-p 0.95 --top-k 20 \
6 -ngl 99
If you try one of these and something breaks or feels off, I'd like to hear about it.
