language: 3-bit, with 4-bit on sensitive down_proj / v_proj plus embed_tokens and lm_head
vision encoder: left in bfloat16 (mlx-vlm skips multimodal modules)
average 3.91 bits/weight, ~12 GB on disk (3 shards)
smoke generate peak 13.7 GB unified memory on an M5 Max

The 4-bit sibling is ~15 GB / 4.70 bpw. Do not use mlx-lm; vision needs mlx-vlm.

Eval

Held-out tool-call split from mars2titan/amiko-openhermit-toolsft: first assistant tool JSON in the first 260 eval sessions, greedy decode, 256 new tokens. First brace-matched JSON object scored (think tags stripped). 4-bit numbers are from that checkpoint's card.

	Base	Hermit 4-bit	Hermit 3-bit
Valid JSON	94/260 (36%)	250/260 (96%)	216/260 (83%)
Tool name exact	56/260 (22%)	156/260 (60%)	142/260 (55%)

JSON format drops more than tool-name choice. 4-bit stays the quality default; 3-bit is the 16 GB Mac build.

Use with MLX

pip install -U mlx-vlm

bash

python -m mlx_vlm generate \
  --model mars2titan/amiko-hermit-1.0-27b-vl-mlx-3bit \
  --max-tokens 256 --temperature 0.0 \
  --prompt "Describe this image." --image <path>

python -m mlx_vlm.server --model mars2titan/amiko-hermit-1.0-27b-vl-mlx-3bit

Specs

Architecture: Qwen3.5 vision-language (Qwen3_5ForConditionalGeneration)
Quant: affine mixed 3/4-bit, group size 64, 3.91 bits/weight, ~12 GB
Train: 1 epoch, 250 steps, QLoRA r=16 α=32. Loss 3.18 → 0.62, token accuracy 47% → 85%
License: Apache 2.0