# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/Kimi-K2.7-Code-MLX-3.6bit

Benchmarks

metric	this model
bpw	3.578
base memory	427.579
peak memory (1024/512)	460.444
prompt tok/s (1024)	218.851 ± 0.208
gen tok/s (512)	21.035 ± 0.049
perplexity	4.462 ± 0.037
arc_challenge	0.692 ± 0.021
hellaswag	0.780 ± 0.019

Methodology

Quantized with a mlx-lm fork. MLX quantization options differ than llama.cpp, but the principles are the same:

Sensitive layers like MoE routing, attention, and output embeddings get higher precision
More tolerant layers like MoE experts get lower precision