This conversion is intended for local inference on Apple Silicon using MLX / MLX-LM. It preserves the upstream model architecture and tokenizer files, while quantizing supported linear weights to 4-bit for a smaller memory footprint.

Attribution and license

Base model: Qwen/Qwen2.5-Coder-3B
Upstream license: Qwen Research License Agreement (qwen-research)
Upstream license file: https://huggingface.co/Qwen/Qwen2.5-Coder-3B/blob/main/LICENSE
This repository is a derivative conversion/quantization and is not the original Qwen release.

A NOTICE file is included in this repository. By using, copying, modifying, redistributing, deploying, or making this derivative model available to others, you are responsible for complying with all applicable upstream terms, including the Qwen Research License Agreement and any additional terms, notices, access requirements, usage instructions, export-control, sanctions, or other legal requirements that apply to the upstream model.

If the upstream license, notices, or model-page terms are updated, those upstream terms may impose additional or different obligations. Please review the upstream model page and license before use or redistribution.

No affiliation, sponsorship, endorsement, or trademark grant

This repository is independently prepared and published by the repository owner. It is not affiliated with, sponsored by, approved by, or endorsed by Alibaba Cloud, Qwen, or their affiliates unless they explicitly state otherwise.

The names "Qwen", "Alibaba Cloud", and related marks are used here only for reasonable descriptive attribution and identification of the upstream base model. No trademark license or other rights in those marks are granted by this repository.

Conversion details

Source: Qwen/Qwen2.5-Coder-3B
Format: MLX / MLX-LM
Quantization: 4-bit affine weight quantization
Group size: 64
Upstream revision: 09d9bc5d376b0cfa0100a0694ea7de7232525803
Conversion command:

bash

python -m mlx_lm.convert \
  --hf-path Qwen/Qwen2.5-Coder-3B \
  --mlx-path Qwen2.5-Coder-3B-Arbor-4bit \
  -q \
  --q-bits 4 \
  --q-group-size 64

Usage

Install MLX-LM:

pip install -U mlx-lm

Example generation:

bash

python -m mlx_lm.generate \
  --model masato25/Qwen2.5-Coder-3B-Arbor-4bit \
  --prompt "Write a Python function that checks whether a string is a palindrome." \
  --max-tokens 256 \
  --temp 0.0

Note: the upstream model is a base/pretrained code language model, not an instruction/chat-tuned model. The upstream model card does not recommend using base language models directly for conversations.

Intended use

This quantized derivative is intended for experimentation, research/evaluation, prototyping, and Apple Silicon local inference where permitted by the upstream Qwen Research License Agreement.

You should evaluate whether your intended use is permitted under the upstream license and related terms. This repository does not expand, waive, or modify any upstream restrictions.

Limitations and safety

Quantization can change output quality, numerical behavior, robustness, and safety characteristics compared with the original model. This repository does not claim improved accuracy, safety, bias mitigation, alignment, or suitability for any particular purpose over the upstream Qwen model.

Model outputs may be inaccurate, unsafe, biased, offensive, incomplete, vulnerable, or otherwise unsuitable for your use case. Do not rely on the model as the sole source of truth for code correctness, security, medical, legal, financial, safety-critical, or other high-stakes decisions. Review and test generated code before use.

Disclaimer

This derivative model is provided as-is and without warranties or conditions of any kind, express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, title, non-infringement, accuracy, availability, or error-free operation.

To the maximum extent permitted by applicable law, the repository owner is not liable for any direct, indirect, incidental, special, consequential, exemplary, punitive, or other damages arising from or related to use of this repository, the derivative model, generated code, or other model outputs.

Nothing in this README is legal advice. You are responsible for reviewing and complying with the applicable license terms and laws.