This repository contains an MLX/oMLX-compatible oQ8 quantization of Qwen/Qwen3.5-2B. It is a derivative quantized release; all credit for the original model architecture, training, and base release belongs to Qwen.

Summary

Base model: Qwen/Qwen3.5-2B
Quantization: oMLX/oQ level 8, affine 8-bit, group size 64
Runtime target: oMLX / MLX
Format: MLX safetensors
Dtype for preserved/unquantized tensors: bfloat16
Multimodal: yes; vision tensors preserved
Native MTP: yes; MTP tensors preserved
License: Apache-2.0, inherited from the base model

Files

text

README.md
LICENSE
chat_template.jinja
config.json
merges.txt
model.safetensors
preprocessor_config.json
tokenizer.json
tokenizer_config.json
vocab.json

Verification

Verified before public release:

model.safetensors opens successfully
safetensors metadata: {"format": "mlx"}
model.safetensors size: 2,735,417,896 bytes
tensor count: 1020
MTP tensor count: 29
vision tensor count: 297
clean private Hugging Face download: pass
clean oMLX download/load from Hugging Face: pass
text smoke test: prompt Reply with exactly OK. returned OK
image smoke test: red square returned red

See PUBLISH_CHECK.md for release verification details.

Use with oMLX/OpenAI-compatible API

Example request once the model is downloaded into an oMLX server:

bash

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.5-2B-oQ8-mtp",
    "messages": [{"role": "user", "content": "Reply with exactly OK."}],
    "max_tokens": 16,
    "temperature": 0
  }'

For vision use, send OpenAI-style multimodal chat content with an image_url item.

Quantization details

json

{
  "model_path": "/Users/dylanrapanan/.omlx/models/Qwen/Qwen3.5-2B",
  "oq_level": 8,
  "group_size": 64,
  "sensitivity_model_path": "",
  "text_only": false,
  "dtype": "bfloat16",
  "preserve_mtp": true,
  "auto_proxy_sensitivity": true
}

Notes

This is not the original base model; it is a quantized derivative for local MLX/oMLX use.
Native MTP is enabled by preserving the model's MTP heads. In oMLX, keep mtp_enabled on; do not confuse this with external VLM drafter settings.
The model was published private-first, then clean-downloaded, tensor-verified, clean-loaded in oMLX, smoke-tested, and only then made public.

Attribution

All credit for the original model goes to Qwen/Alibaba Cloud. This repository only provides a quantized MLX/oMLX artifact for convenience.

dylpickle13/Qwen3.5-2B-oQ8-mtp

Qwen3.5-2B-oQ8-mtp