You can deploy this model and start earning money today!

trymirai/
Qwen3.5-4B-L

Name: Qwen3.5-4B-L
Availability: InStock

Views

No views yet

Mirai's Qwen3.5-4B Large Quantization

A large, high-quality quantization of Qwen3.5-4B prepared for efficient local inference on Apple silicon.

Mirai Labs

KL divergence versus language-model checkpoint size

Evaluation data mixture: 45% public agentic, 30% public SFT/long-context, 25% private chat data.

Quickstart

If you are on macOS, the easiest way is to install the mirai Homebrew package and then run the CLI:

brew install mirai
mirai --model trymirai/Qwen3.5-4B-L

Currently only Apple silicon inference is supported. If you want to build things from source, read this overview.

Learn more

Method

Mirai Large uses 8-bit symmetric integer quantization with bfloat16 scales and group size 32. Block-diagonal Random Hadamard Transforms with block size 32 are used to reduce activation and weight outliers. The checkpoint was prepared with post-training quantization.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex

@misc{mirai-quant,
    title  = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
    author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
    note   = {In collaboration with others at Mirai Labs},
    month  = {June},
    year   = {2026},
    url    = {https://trymirai.com/blog/quantization}
}

Original model

This is a quantized version of Qwen/Qwen3.5-4B. For architecture details, intended use, evaluations, and limitations, see the original model card.