This repository contains the 4-bit NVFP4 quantized weights for Qwen/Qwen2.5-Coder-3B, optimized for low-latency inference on Apple Silicon using the oMLX framework.

Qwen2.5-Coder-3B is the ultra-lightweight entry in the Qwen2.5 coding specialist series. Despite its exceptionally compact 3 billion parameter footprint, it inherits the advanced architectural and training enhancements of the broader Qwen2.5-Coder family, making it uniquely suited for fast, edge-based autocomplete, inline code generation, and low-resource deployments.

🚀 Efficiency & Performance Advantages

By combining the highly efficient 3B parameter base model with a 4-bit NVFP4 quantization mapping, this variant achieves:

⚡ Blazing-Fast Generation (TPS): Exceptional token generation and prefill speeds, allowing for near-instantaneous IDE code completions.
📉 Minimal Memory Footprint: Extremely small VRAM utilization, freeing up system resources to comfortably run alongside heavy local developer environments.
⚙️ Seamless Mac Optimization: Native acceleration when coupled with modern execution layers like oMLX on Apple Silicon.

🛠️ Deployment & Execution Quickstart

To utilize this model on macOS, ensure you are running an inference wrapper configured to handle nvfp4 metadata structures.

Running with `oMLX`

Execute local evaluation benches natively via terminal: omlx bench --model your-hf-username/Qwen2.5-Coder-3B-MLX-nvfp4 --prompt "Write a Python function to clear a list."

bkideas/Qwen2.5-Coder-3B-MLX-nvfp4

Qwen2.5-Coder-3B-MLX-nvfp4

🚀 Efficiency & Performance Advantages

🛠️ Deployment & Execution Quickstart

Running with oMLX

bkideas/
Qwen2.5-Coder-3B-MLX-nvfp4

Running with `oMLX`