Kimi K2.5 optimized to run even more comfortably on a Mac Studio M3 512G.
My 2.8 bit quants fit into 380G memory.
This 2.5 bit one hovers around 350G, while matching the original 2.8 bit quant in quality.
The main motivation to compress even further was to support a full "Claude Code in a box" system, which requires not just an
Opus replacement (Kimi K2.5) but also Haiku and Sonnet replacements (Qwen 3.5) for background tasks and subagents.
Usage
sh
1# Start server at http://localhost:8080/v1/chat/completions
2uvx --from mlx-lm --with tiktoken \
3 mlx_lm.server \
4 --host 127.0.0.1 --port 8080 \
5 --trust-remote-code \
6 --model spicyneuron/Kimi-K2.5-MLX-2.5bit
78# Kimi K2.5 requires tiktoken + remote code for the tokenizer
Methodology
Quantized with a mlx-lm fork, drawing inspiration from Unsloth/AesSedai/ubergarm style mixed-precision GGUFs.
MLX quantization options differ than llama.cpp, but the principles are the same:
Sensitive layers like MoE routing, attention, and output embeddings get higher precision (BF16, 8, 4)
More tolerant layers like MoE experts get lower precision (2, 3)
This one is much smaller than Unsloth's UD-Q2_K_XL
in size, and loads and runs noticeably faster thanks to MLX.
