# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/GLM-5.2-MLX-4.5bit

Benchmarks

metric	this model
bpw	4.535
base memory	392.454
peak memory (1024/512)	422.787
prompt tok/s (1024)	194.114 ± 0.079
gen tok/s (512)	17.781 ± 0.028
kl mean*	0.049 ± 0.002
kl p95	0.113 ± 0.002
perplexity	4.642 ± 0.036
arc_challenge	0.690 ± 0.021
hellaswag	0.780 ± 0.019

* KL calculated against the largest quant I could run locally (~5.3 bit). Real KL is against FP will be higher.

Methodology

Quantized with a mlx-lm fork. MLX quantization options differ than llama.cpp, but the principles are the same:

Sensitive layers like MoE routing, attention, and output embeddings get higher precision
More tolerant layers like MoE experts get lower precision