Calibrated 2-bit MLX quantization of tencent/Hy3 (Hunyuan 3.0, 295B-A21B MoE),
produced with omlx oQe at level 2. Effective 2.37 bits/weight over quantizable
weights, 87.7 GB on disk. Targets Apple Silicon.
This is a shell-reduced variant of mlx-community/Hy3-oQ2e: the routed experts
are identical, but the non-expert layers (attention, embeddings, lm_head) are
quantized below the 8-bit that oQ2e keeps, trading a small amount of quality for
~2 GB.
Quantization layout
Component
oQ2e (parent)
This model (2.37 bpw)
Routed experts (98%)
2-bit gs128 + imatrix
2-bit gs128 + imatrix
Attention
8-bit gs64
6-bit gs128
Embeddings / lm_head
8-bit gs64
4-bit gs128
Routed experts are bit-for-bit the same as oQ2e (imatrix reused from the same
calibration cache). Only the shell changed.
How it was quantized
Built from the BF16 source (~550 GB / 591 GiB) with omlx oQ level 2 + imatrix
weighting. To stay within 128 GB RAM: the sensitivity pass reused the existing
oQ2 quantization instead of building a full-precision proxy, and the importance
matrix was reused from the oQ2e calibration cache. Streamed tensor-by-tensor.
Requirements
Runs natively in oMLX. For mlx-lm, hy_v3 support is pending upstream; until then:
oMLX intelligence suite, 300 seeded samples per benchmark, identical questions
across models. I ran seeded samples — this is not a complete benchmark run,
so read the differences as noise and test the versions against your own workload
before picking one.
