This is a standard GGUF that runs on stock llama.cpp.
Variants
Variant
File
Size
BPW
14 GB v2
KAT-Coder-V2.5-Dev-Cerebellum-14GB-v2.gguf
12.1 GB
2.93
Quantization recipe
Base model: Kwaipilot/KAT-Coder-V2.5-Dev (Qwen3.6-35B-A3B, 35B total / ~3B active MoE)
Source format: BF16 merged GGUF
Imatrix: KAT-specific lite coder imatrix built from HumanEval+ / MBPP+ samples
Base quant: Q3_K_M
Key override: expert ffn_down weights in layers 20-39 promoted from Q2_K to Q3_K
Benchmarks
Measured on an RTX 3090 with llama-server -ngl 99 --parallel 4 -c 24576 --reasoning off --reasoning-budget 0.
Benchmark
v2 (12.1 GB)
maxx-v4 (15.6 GB)
Q5_K_M pure (24.0 GB)
Previous 35B A3B Cerebellum 14 GB
HumanEval+ chat base
92.07%
90.85%
90.85%
89.63%
HumanEval+ chat plus
89.02%
87.80%
87.80%
85.98%
BigCodeBench hard
28.05%
27.0%
27.7%
25.70%
v2 is the best coding quant we have produced for this architecture. Both larger experiments (16 GB mixed-precision maxx-v4 and 24 GB pure Q5_K_M base) scored lower, confirming the coding-specific imatrix + targeted late-layer ffn_down promotion in v2 extracts more performance per gigabyte than simply raising the base quant.
Coding ablation
A per-group ablation (demote one group to Q2_K over a Q4_K_M base, measure HumanEval delta) found every tensor group is coding-critical. The largest drops came from attn_q (−76.2%), ffn_up_all (−73.2%), and ssm_beta (−71.9%). Even the "least damaging" group, attn_qkv, dropped coding performance by 56.7%.
This means v2 is close to the practical floor for this model at ~14 GB. Further gains require a higher starting base quant or layer-level drilling, not blanket demotion.
Full logs, samples, eval outputs, ablation data, and one-shot game generations are in benchmark_results/.
Experimental result files for maxx-v4 and the rejected Q5_K_M pure base are also in benchmark_results/ for comparison.
Text-only. The base model tag includes image-text-to-text, but the published source weights used here are the text-only KAT-Coder fine-tune. Vision was not tested.
Native context is 262,144 tokens. On a 24 GB RTX 3090, 24K–98K context is the practical daily-driver range.
