Qwen3.8-27B-oQ6e-mtp

• Qwen3.8-27B-oQ4e-fp16-mtp — 4-bit / fp16, ~16.7 GiB, tuned for M1/M2; ANE-eligible in oMLX 0.6.1 (+21% prefill measured on M2 Ultra)
• Qwen3.8-27B-oQ8e-fp16-mtp — 8-bit / fp16, ~27.5 GiB, higher fidelity if you have the memory
• Qwen3.8-27B-oQ4e-mtp — 4-bit / bf16, ~17 GB, for 32 GB-class machines or long-context use on 64 GB
• Qwen3.8-27B-oQ8e-mtp — 8-bit / bf16, ~27.9 GiB; same quantized tensors and imatrix as the oQ8e-fp16 build, bf16 floating tensors for M3/M4+ numerical safety

Quantization details

• Model type: qwen3_5
• Bits: 6 (effective 6.8 bpw mixed precision), group size 64 — ~22 GiB weights, leaving KV-cache headroom under a 64 GB machine's default GPU-memory limit
• Enhanced quantization (oQe): imatrix-calibrated (1024 samples) — affine quantization weighted by activation importance
• MTP weights preserved (mtp.* tensors + config) — multi-token-prediction / Lightning MTP works after quantization
• Non-quant weight dtype: bfloat16 (matches the base model; the safe choice on M3/M4/M5)
• Vision components included (not a text-only strip)
• Format: MLX safetensors

Recommended sampling (per the Qwen3.8 model card)

• enable_thinking (default true)
• preserve_thinking (default true) — keeps reasoning traces across multi-turn history
• reasoning_effort: xhigh (default) / medium / low — in our testing (on the 8-bit sibling), medium reduced thinking volume ~25% with no loss on agentic tasks

Tested