A 4-bit mixed-precision MLX quant of Qwen/Qwen3.6-27B. Per-layer bit-widths come from a KL-divergence sensitivity pass on a six-domain calibration mix (prose · reasoning · code · agent · tool-call · constraint-bearing instructions). Sensitive layers go to 8-bit; robust ones stay at 4-bit. The on-disk size is within ~5 % of a stock uniform 4-bit MLX quant.

Quantization details

Property	Value
Predominant precision	4-bit
Layers at 8-bit (sensitive)	220
Layers at 4-bit (robust)	276
Total quantized layers	496
Group size	64
Calibration mix	six-domain mix (40 samples × 6 domains)
Reference for sensitivity	bf16 (auto-resolved; falls back to uniform-4-bit if bf16 doesn't fit)

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average. The mixed allocation is what lets this build beat stock uniform-4-bit at the same disk size. Benchmark deltas are below.

Usage

Load it with mlx-lm and use it as usual:

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen3.6-27B-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=200,
)

For more (mixed-precision KV-cache serving, sensitivity-aware LoRA fine-tuning, OpenAI + Anthropic-compatible inference server, hot-swap mounted adapters, sandboxed Python execution for agent workflows), install mlx-optiq:

pip install mlx-optiq

See the Qwen3.6 family guide on mlx-optiq.com for sampling defaults, training recipes, and family-specific caveats.

Benchmarks

Five-metric suite that drives the Capability Score:

Metric	Score
MMLU (5-shot, 1000 samples)	87.4%
GSM8K (1000 samples, 3-shot CoT)	92.0%
IFEval (full set, strict)	74.1%
BFCL-V3 simple (200 single-turn calls)	74.0%
HumanEval (164 problems, pass@1)	90.2%
Capability Score (mean of the 5 benchmarks above)	83.6
KL vs uniform-4-bit reference (mean / p95)	0.0895 / 0.4101
On-disk size	17.5 GB

The Capability Score is the simple unweighted mean of the five benchmarks. Every metric gets one equal vote. Disk size is reported next to it as an honest second axis instead of being folded into the score. See the eval-framework writeup for the full methodology.

License

Apache 2.0 (inherits from base model).

Meguriai/Qwen3.6-27B-OptiQ-4bit

mlx-community/Qwen3.6-27B-OptiQ-4bit

Quantization details

Usage

Benchmarks

Links

License

Meguriai/
Qwen3.6-27B-OptiQ-4bit