You can deploy this model and start earning money today!

anerjy/
Qwen3.5-2B-OptiQ-4bit-text-mlx

Name: Qwen3.5-2B-OptiQ-4bit-text-mlx
Rating: 2.5 (128 reviews)

mlx-community/Qwen3.5-2B-OptiQ-4bit

quantized

apache-2.0

4-bit

Views

No views yet

Qwen3.5-2B-OptiQ-4bit — text-only (vision tower removed)

Derived from mlx-community/Qwen3.5-2B-OptiQ-4bit (sensitivity-aware mixed 4/8-bit OptiQ quantization of Apache-2.0 Qwen/Qwen3.5-2B).

Surgery for on-device text-only use (e.g. iPhone via mlx-swift-lm):

dropped the vision tower (optiq/optiq_vision.safetensors, 663 MB) and the MTP head
renamed all 694 text tensors: language_model. prefix stripped → flat model.* layout
flattened config.json (text_config → top level), model_type: qwen3_5_text (matches mlx-swift-lm LLMModelFactory's flat text class)
remapped all 187 per-layer OptiQ quantization overrides to the renamed module paths (this preserves the mixed 4/8-bit allocation)

Verified with the same mlx-swift-lm stack used on device: 51/52 on a 52-item Chinese assistant-routing benchmark (structured JSON slot-filling, temp=0, repetition_penalty 1.05 / repetition_context 256), JSON validity 52/52.

Note: Python mlx_lm cannot load this repo — its qwen3_5 implementation expects the VLM layout (language_model. prefix). This flat layout targets mlx-swift-lm's qwen3_5_text class (registered in LLMModelFactory).

anerjy/Qwen3.5-2B-OptiQ-4bit-text-mlx

Qwen3.5-2B-OptiQ-4bit — text-only (vision tower removed)

anerjy/
Qwen3.5-2B-OptiQ-4bit-text-mlx