Sensitivity-guided mixed-precision quantization of
Qwen/Qwen3.6-35B-A3B. Cerebellum measures which weight groups survive extreme compression and which don't, then writes a single GGUF with per-tensor precision assignments — a standard GGUF that runs on stock
llama.cpp, no fork.
v3 (11 GB) is the tightest-VRAM build. The 14 GB spends ~3 GB more to promote the routed ffn_down_exps to Q4_K — the group the ablation identifies as where the quality lives — and that gives it the family's best coding plus 160K+ context headroom. It posts above the 16 GB uniform Q3_K_M (−2 GB) and matches the 17.3 GB Base (−3.3 GB): the Base's extra promotions buy ~0 coding, so 14 GB is the efficient point. Pick v3 only when VRAM is tight or you need the vision projector.
1# 14 GB (recommended)
2llama-server -m Qwen3.6-35B-A3B-Cerebellum-14GB.gguf -ngl 99 -fa on --reasoning off
3
4# v3 (smallest, with vision)
5llama-server -m Qwen3.6-35B-A3B-Cerebellum-v3-Q3_K_M.gguf --mmproj mmproj-F16.gguf -ngl 99 -c 8192
Built with
Cerebellum — sensitivity-guided mixed-precision quantization: crush each tensor group, measure the impact, allocate precision under a size budget, output a plain GGUF. imatrix-calibrated. Quantized by
@deucebucket.
This line has a recorded data point in
club-3090's BENCHMARKS (author-rig numbers from a full
report.sh --full chain). The same report corrected their engine-support table for this model (
issue #390,
PR #393). Numbers there are author-reported, not club-validated.