Asymmetric GGUF quantization of
Kwaipilot/KAT-Coder-V2.5-Dev
(qwen35moe: 34.7B total / ~3B active, 40 layers, 256 routed experts top-8,
1 shared expert, GDN hybrid attention, 262144 ctx), built for the
bmk bare-metal inference engine.
Importance-matrix calibrated (Kwaipilot imatrix, 510 entries / 802 chunks).
File size: 10.9 GiB. Designed for 8/12/16 GB VRAM cards via expert-granular
streaming (only the top-8 routed experts per layer are loaded).
1bash download_model.sh bmk-q2
2LD_LIBRARY_PATH=vendor/cuda-runtime ./bmk -m models/kat-coder-asym.gguf -p "..." -n 64
llama.cpp-compatible GGUF; the tensor layout follows the standard
qwen35moe architecture.
