Mirai-M is on the size-KL Pareto frontier: we found no checkpoint that is smaller while also having lower KL divergence.
1brew install mirai
2mirai --model trymirai/Qwen3.6-27B-M
Currently only Apple silicon inference is supported. If you want to build things from source, read this
overview.
Mirai Medium uses 4-bit asymmetric integer quantization with 4-bit zero points, bfloat16 scales, and group size 64. Block-diagonal Random Hadamard Transforms are used to reduce activation and weight outliers. The checkpoint was prepared with post-training quantization followed by quantization-aware distillation.
In quantization and model-size tier, this checkpoint is comparable to Unsloth's
Q4_K_M GGUF.
If you find our work helpful, feel free to give us a cite.
1@misc{mirai-quant,
2 title = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
3 author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
4 note = {In collaboration with others at Mirai Labs},
5 month = {June},
6 year = {2026},
7 url = {https://trymirai.com/blog/quantization}
8}
This is a quantized version of
Qwen/Qwen3.6-27B. For architecture details, intended use, evaluations, and limitations, see the
original model card.
