A quality-first, mixed-precision MLX quantization of Qwen3.6-35B-A3B (35B-total / 3B-active hybrid MoE: 30 GatedDeltaNet + 10 full-attention layers, 256 experts top-8 + shared expert). Runs natively on Apple Silicon via mlx-lm and mlx-swift-lm — standard MLX affine format, no custom kernels.

Quantization recipe

Bits are allocated by measured per-tensor-class sensitivity, not uniformly. Routed experts (~90% of parameters) carry the 4-bit budget; every tensor that fires on every token is protected at higher precision.

Tensor class	Bits / group	Added rel. RMS vs BF16
Routed experts (`switch_mlp`, 256×40)	4 / 64 + per-group MSE-optimal clip	8.90%
Shared expert (×40)	6 / 64	2.53%
DeltaNet projections (`in_proj_qkv/z`, `out_proj`, ×30)	6 / 64	2.43%
Full attention (`q/k/v/o`, ×10)	6 / 64	2.40%
`embed_tokens` / `lm_head` (untied)	5 / 64	4.46% / 4.62%
Router `mlp.gate`, `shared_expert_gate`, conv1d, DeltaNet a/b, norms	bf16 (unquantized)	0

20.0 GB weights (~4.6 bpw effective). The MSE-clip pass re-quantizes each expert group with a 16-step clip-factor search scored by the exact MLX affine quantizer (bf16 scale/bias storage included), improving experts from 9.36% → 8.90% at identical size and format.

Provenance

Quantized from the official FP8 release (Qwen/Qwen3.6-35B-A3B-FP8, e4m3 + 128×128 block scales), streaming-dequantized to a BF16 master (bit-exact validated), then converted with mlx_lm.convert using a custom per-tensor quant_predicate. Note this inherits the FP8 release's ~2.7% quantization floor relative to the BF16 original.

Text-only: the vision tower and the MTP (multi-token-prediction) head are not included (dropped by the mlx-lm text serving path).

Benchmarks (Apple M5 Max, 40-core GPU, 614 GB/s, 128 GB)

Metric	Value
Decode, single stream (mlx-lm)	~132 tok/s
Prefill @ 2048 tokens (MLX-Swift)	~3,180 tok/s
Batched decode aggregate B=6 (MLX-Swift scheduler)	~254 tok/s
Peak memory (generation)	20.2 GB

For reference, the same machine runs the popular 21 GB community MLX build at ~123 tok/s single-stream — this artifact is ~5% smaller and ~6% faster, with a fully documented recipe.

Usage

bash

pip install mlx-lm
python -m mlx_lm generate --model EigenLabs/Qwen3.6-35B-A3B-MLX-mixed-4bit \
  --prompt "Explain Rayleigh scattering in two sentences." --max-tokens 256

python

from mlx_lm import load, generate
model, tokenizer = load("EigenLabs/Qwen3.6-35B-A3B-MLX-mixed-4bit")
print(generate(model, tokenizer, prompt="Hello", max_tokens=100))

Works out of the box with LM Studio and MLX-Swift apps (loads via LLMModelFactory / qwen3_5_moe).

Evaluation status

Reported numbers are weight-space reconstruction errors and throughput benchmarks. Perplexity / task-eval comparisons against the BF16 teacher are in progress; treat this as a well-instrumented engineering release rather than an eval-certified one.