You can deploy this model and start earning money today!

majentik/
Qwen2.5-1.5B-Instruct-MLX-4bit

Name: Qwen2.5-1.5B-Instruct-MLX-4bit
Availability: InStock

Qwen/Qwen2.5-1.5B-Instruct

quantized

apache-2.0

4-bit

Views

No views yet

Qwen2.5-1.5B-Instruct-MLX-4bit

Plain affine 4-bit (group-size 32) MLX quantization of Qwen/Qwen2.5-1.5B-Instruct. This is the baseline sibling of the DWQ pack — for higher quality at the same size and bit-width, use the DWQ variant (perplexity 9.45 vs 9.86 here).

Variants

Variant	Size	Status
Qwen2.5-1.5B-Instruct-MLX-8bit	1.5 GB	teacher
Qwen2.5-1.5B-Instruct-MLX-4bit	0.93 GB	this repo — plain affine 4-bit (gs32)
Qwen2.5-1.5B-Instruct-MLX-4bit-DWQ	0.93 GB	DWQ (recommended)

Reproduce

bash

python -m mlx_lm convert --hf-path Qwen/Qwen2.5-1.5B-Instruct \
  --mlx-path Qwen2.5-1.5B-Instruct-MLX-4bit -q --q-bits 4 --q-group-size 32

Group-size 32 per docs/quantization-policy.md rule 1 (doubles tunable scale/bias params at low bit width).

Usage

bash

pip install mlx-lm
python -m mlx_lm generate --model majentik/Qwen2.5-1.5B-Instruct-MLX-4bit \
  --prompt "The capital of France is"

License

Apache-2.0, inherited from the base model Qwen/Qwen2.5-1.5B-Instruct.