This is an oQ5 MLX quantized derivative of
tencent/Hy-MT2-30B-A3B.
The model was quantized locally with oMLX oQ5. The resulting config targets approximately 5.61 bpw and includes a packaged hy_v3.py model file so MLX-LM can load the HYV3 architecture from the model directory.
1mlx_lm load: passed
2generation smoke test: passed
3prompt: Hello
4max tokens: 4
5peak memory: 19.651 GB
1python -m mlx_lm generate \
2 --model /path/to/Hy-MT2-30B-A3B-oQ5-MLX \
3 --ignore-chat-template \
4 --prompt "Hello" \
5 --max-tokens 32 \
6 --temp 0
The base model is distributed under the Tencent HY Community License Agreement. This distribution includes LICENSE.txt and NOTICE from/for the Tencent HY license requirements.
This repository is not affiliated with, sponsored by, or endorsed by Tencent.
