This model was converted using mlx-vlm from the pc/fix-qwen35-predicate branch, which includes fixes for Qwen3.5 model support (proper handling of MoE gate layers, shared_expert_gate, and A_log casting).

Conversion command:

bash

python3 -m mlx_vlm convert \
  --hf-path "Qwen/Qwen3.5-4B" \
  --mlx-path "./Qwen3.5-4B-MLX-4bit" \
  -q --q-bits 4 --q-group-size 64

Important Note

A better, more optimized conversion may be available from @Prince (@Blaizzy) in the MLX VLM community. Check the mlx-community organization for updated versions as official Qwen3.5 support is merged into the main mlx-vlm branch.

Related Models

bf16 (full precision): mlx-community/Qwen3.5-4B-MLX-bf16
8-bit quantized: mlx-community/Qwen3.5-4B-MLX-8bit
Original: Qwen/Qwen3.5-4B

Usage

python

from mlx_vlm import load, generate

model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")

output = generate(
    model,
    processor,
    prompt="Describe this image.",
    image="path/to/image.jpg",
    max_tokens=512
)
print(output)

CLI:

bash

python3 -m mlx_vlm.generate \
  --model mlx-community/Qwen3.5-4B-MLX-4bit \
  --image path/to/image.jpg \
  --prompt "Describe this image."

License

This model inherits the Apache 2.0 license from the original Qwen model.

wabibito/Onyx-Qwen3.5-4B-4bit

Qwen3.5-4B-MLX-4bit

Model Details

Conversion Details

Important Note

Related Models

Usage

License

wabibito/
Onyx-Qwen3.5-4B-4bit