This is a
2-bit MLX TextOnly quantization of
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16, optimized for inference on Apple Silicon.
The language-model weights were converted to MLX and quantized to 2-bit. The vision components of the original multimodal model are not included, making this release TextOnly and reducing memory requirements.
1mlx_lm.generate \
2 --model EgorKodin/Qwen3.8-27B-ABLITERATED-2bit-MLX-TextOnly \
3 --prompt "Hello! Explain quantization briefly." \
4 --max-tokens 512
1mlx_lm.generate \
2 --model EgorKodin/Qwen3.8-27B-ABLITERATED-2bit-MLX-TextOnly \
3 --prompt "Hello!" \
4 --max-tokens 512 \
5 --chat-template-config '{"enable_thinking": false}'
2-bit quantization is aggressive and prioritizes low memory usage and compact model size. Depending on the task, quality may be lower than the 3-bit or higher-bit versions.
This repository provides an MLX quantization/conversion of the original model. Credit for the underlying model and abliteration work belongs to the original authors.
