KAT-Coder-V2.5-Dev REAP-50 NVFP4A16 — GPTQ rounding variant
Published for completeness and independent verification, not as a
recommended alternative. If you're choosing a checkpoint, use
Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16
— that is the primary release, actively maintained, and this card exists
specifically to document why this variant is
not a better choice.
Architecture
1graph TD
2 Base["Kwaipilot/KAT-Coder-V2.5-Dev<br/>Qwen3.5 MoE - 256 experts - ~69 GB bf16"]
3 subgraph Build ["Build - RTX 5070 Ti, SM120"]
4 REAP["REAP expert prune 50% 256 -> 128 experts + router-renorm fix"]
5 Strip["strip vision tower + 333 untrained tensors"]
6 Quant["NVFP4A16 quantize (weight-only, data-free, 82 s)"]
7 end
8 subgraph HF ["Published formats"]
9 A16["REAP-50-NVFP4A16 - 12.45 GiB (default, vLLM)"]
10 W4A4["REAP-50-NVFP4-W4A4 (native FP4)"]
11 GPTQ["REAP-50-NVFP4A16-GPTQ (null result, kept for transparency)"]
12 GGUF["REAP-50-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0)"]
13 BF16["REAP-50-bf16 (pruned source)"]
14 end
15 Bench["A16 single draws - HumanEval+ 89.0% - MBPP+ 90.5% - SWE-bench Verified 52.0%"]
16 Base --> REAP --> Strip --> Quant --> A16
17 Strip --> BF16
18 BF16 -. re-quant .-> W4A4
19 BF16 -. re-quant .-> GPTQ
20 BF16 -. convert .-> GGUF
21 A16 --> Bench
What this is
Same REAP-50%-pruned source checkpoint as the primary release, quantized
to the identical NVFP4A16 scheme (weight-only, 4-bit) — but with GPTQ
(Hessian-based, error-correcting) rounding instead of the primary release's
plain round-to-nearest (RTN). Same size, same architecture, same serving
kernel. The only variable changed is the quantization algorithm.
12.4512 GiB — exact byte-for-byte size match to the primary release.
Why this was tried
GPTQ is documented in current literature to generally outperform RTN for
NVFP4 weight recovery. It was tested here specifically to see whether that
general finding held for this checkpoint and closed any of the gap to the
competitive bar (Devstral Small 2512, 56.4% SWE-bench Verified under the
same mini-swe-agent bash-only scaffold).
Result: no measurable difference from the primary release
Paired McNemar test (not just comparing point estimates) against the
primary release's own accuracy suite, same evaluation methodology:
| benchmark | primary (RTN) | this build (GPTQ) | discordant pairs | McNemar p |
|---|
| HumanEval+ | 90.85% | 89.63% | 6 / 164 | 0.68 (not significant) |
| MBPP+ | 89.95% | 89.42% | 18 / 378 | 0.81 (not significant) |
Both comparisons are underpowered (well under the ~40-50 discordant pairs
needed for adequate statistical power) — this is genuinely "no detectable
difference," not confident evidence of true equivalence, and not evidence
GPTQ is worse either. The run itself was clean: 5h23min, zero exceptions,
zero fallback-to-RTN warnings across all 15,520 quantized modules.
Practical read: this specific checkpoint's RTN baseline appears to
already be close to its own accuracy ceiling, leaving little room for a
better rounding algorithm to show up. GPTQ's general advantage over RTN
(well-supported in the literature for other models) doesn't disappear —
it just isn't visible here at the precision this evaluation can resolve.
Full writeup, including the literature verification done before running
this and the exact promotion criteria this didn't clear:
ROADMAP.md
in the primary release's GitHub repo.
Quantization details
| Field | Value |
|---|
| Base | Same REAP-50%-pruned checkpoint as the primary release |
| Quantization method | llm-compressor GPTQModifier — actorder="static", block_size=128, dampening_frac=0.01 |
| Quantization scheme | NVFP4A16 — weight-only, identical to the primary release |
| Calibration | evol-codealpaca (same set, same seed as the primary release, isolating the algorithm as the only variable) |
License
Apache 2.0, inherited from the base model Kwaipilot/KAT-Coder-V2.5-Dev.
