You can deploy this model and start earning money today!

TheAndyRoberts/
Qwen2.5-3B-Instruct-4bit-mlx

Name: Qwen2.5-3B-Instruct-4bit-mlx
Availability: InStock

Views

No views yet

Qwen2.5-3B-Instruct-4bit-mlx

MLX 4-bit quantization of Qwen/Qwen2.5-3B-Instruct, converted with mlx-lm for Apple Silicon.

bash

pip install mlx-lm
mlx_lm.generate --model TheAndyRoberts/Qwen2.5-3B-Instruct-4bit-mlx \
  --prompt "Your prompt here" --max-tokens 100

Apache 2.0 (inherited from base model)