A sensitivity-fenced, mixed-precision 4-bit quantization of Qwen/Qwen3.6-27B for Apple Silicon (MLX). Most of the network runs at 4-bit for speed and size, while the handful of weights that are genuinely sensitive to low-bit quantization are kept at 6/8-bit or BF16.

5.36 bits/weight, ~17 GB on disk
~30 tok/s decode, ~18 GB peak memory on an Apple M5 Max
Weight-only affine quant (activations stay FP16) → no activation-outlier failure modes
Beats a naïve uniform-4bit build on every fidelity metric (see below)

TL;DR on choosing a build: if you have the memory, the 6-bit build is effectively lossless. This mixed-4bit build is for when you want maximum decode speed and the smallest footprint and can accept a measurable (~5% perplexity) quality cost.

Why mixed precision?

Qwen3.6-27B is a hybrid model: 48 of its 64 layers use Gated-DeltaNet linear attention (a recurrence that carries a state across the sequence) and 16 use full softmax attention. We profiled the int4 reconstruction error of all 496 linear tensors. The finding: int4 error is remarkably uniform (~0.091 relative) everywhere except a small set of tensors — the DeltaNet decay/control gates (in_proj_a/in_proj_b) and the GQA k/v projections. Those feed the recurrent state, where a small error compounds over the sequence. So we quantize the bulk to 4-bit and fence exactly those tensors at higher precision — at negligible size cost, since they are tiny.

Precision map

Module	Precision	Rationale
MLP `gate/up_proj` (all 64 layers)	4-bit	42% of params; int4-robust
MLP `down_proj` (middle layers)	4-bit	robust (weight-only quant)
DeltaNet `in_proj_qkv`	4-bit	bulk projection, robust
MLP `down_proj` (first/last ⅛ of layers)	6-bit	slightly worse at network edges
DeltaNet `in_proj_z`, `out_proj`	6-bit	gate / output of the linear-attention path
Full-attn `q_proj`, `o_proj`	6-bit	only ¼ of layers; cheap to protect
Full-attn `k_proj`, `v_proj`	8-bit	6:1 GQA amplifies shared k/v error
`embed_tokens`, `lm_head`	8-bit	untied 248K-vocab; logit precision
DeltaNet `in_proj_a`, `in_proj_b`	BF16 (unquantized)	decay/control gates → recurrence error compounds
`conv1d`, `A_log`, `dt_bias`, RMSNorms	BF16	not linear layers; precision-critical for the SSM state

Resulting distribution by parameter mass: ~68% 4-bit, ~22% 6-bit, ~10% 8-bit, <1% BF16.

Measured quality (vs BF16)

Perplexity, KL-divergence, and top-1 agreement against the BF16 model on a fixed 650-token multi-domain passage. This is an indicative smoke test, not a full benchmark suite — treat it as a relative comparison between builds, not an absolute quality guarantee. Hard-reasoning/long-context degradation is typically larger than perplexity suggests.

Build	bpw	size	ppl	Δppl vs BF16	KL(bf16‖q)	top-1 agree
BF16 (reference)	16	55.6 GB	6.622	—	—	—
8-bit	8.5	27 GB	6.635	+0.20%	0.0018	97.7%
6-bit	6.5	21 GB	6.595	−0.41%¹	0.0087	96.6%
mixed-4bit (this model)	5.36	17 GB	6.977	+5.36%	0.0405	91.5%
uniform-4bit	4.50	15 GB	6.989	+5.53%	0.0651	89.8%

¹ within noise — 6-bit is statistically indistinguishable from BF16 on this sample.

The fencing cuts KL-divergence by ~~38% vs uniform-4bit for +0.86 bpw. Note that because the MLP (61% of params) is 4-bit in both, weight-only fencing cannot close the fundamental 4-bit "cliff" (~~+5% ppl); doing that requires distillation-based quant (DWQ) or rotations.

Usage

bash

pip install mlx-lm
mlx_lm.generate --model EigenLabs/Qwen3.6-27B-MLX-mixed-4bit \
  --prompt "Explain gated delta networks in two sentences." --max-tokens 256

python

from mlx_lm import load, generate
model, tokenizer = load("EigenLabs/Qwen3.6-27B-MLX-mixed-4bit")
messages = [{"role": "user", "content": "What is 17*23?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Qwen3.6 is a reasoning model — the chat template opens a <think> block, so responses begin with a chain of thought.

Requirements & notes

Apple Silicon with MLX; ~20 GB free unified memory (weights + KV). Comfortable on 32 GB+, ideal on 48 GB+ for long context.
Text-only. The MLX qwen3_5 implementation does not load the model's MTP head (so no built-in speculative decoding) or its vision tower (use mlx-vlm separately for images).
The hybrid Gated-DeltaNet layers run on the Metal GPU (no CPU fallback observed).

How it was made

Quantized with mlx_lm.convert using a custom mixed-precision predicate (asymmetric affine, group_size=64):

python

def predicate(path, module):
    if "in_proj_a" in path or "in_proj_b" in path:      return False          # keep BF16
    if "embed_tokens" in path or "lm_head" in path:     return bits(8)
    if "k_proj" in path or "v_proj" in path:            return bits(8)
    if "in_proj_z" in path or "out_proj" in path:       return bits(6)
    if "q_proj" in path or "o_proj" in path:            return bits(6)
    if "down_proj" in path and edge_layer(path):        return bits(6)
    return bits(4)                                                            # MLP bulk, in_proj_qkv

Tokenizer, chat template, and generation config are copied verbatim from the base model.

License & attribution

Apache 2.0, inherited from Qwen/Qwen3.6-27B (© Alibaba / Qwen team). This is a quantized derivative produced by EigenLabs. Please cite the original Qwen3.6 model.