Vector-quantized (trained codebooks + GPTQ compensation) MLX build of Qwen/Qwen3.6-35B-A3B — the quality-first sibling of Qwen3.6-35B-A3B-MLX-VQ-2.6bpw. ~15.0 GB, experts uniformly VQ-3bit (d=4, K=4096 Qwen-native codebook).

⚠️ Not loadable by stock mlx-lm / LM Studio / Ollama / oMLX — use the bundled loader + Metal kernel in code/ (pure Python, mlx>=0.31, no build step).

Quality (PPL vs bf16, ja-think / multilingual holdouts)

build	size	think-ja	multilingual
GPTQ-3.5bpw (scalar)	15.35 GB	+6.6%	+6.9%
this (VQ-3.4bpw)	15.0 GB	+6.0%	+5.8%
VQ-2.6bpw	11.53 GB	+12.4%	+12.2%
GPTQ-2.7bpw (scalar)	12.03 GB	+17.5%	+15.4%
VQ-2.4bpw — for 16 GB Macs	10.49 GB	+19.5%	+21.6%

Smaller and better than the scalar 3.5bpw build on both yardsticks — the trained codebook's edge, now at the quality end of the curve. (Measured with an RTN spine; the shipped artifact's spine is GPTQ-compensated, i.e. slightly better than the table.)

Recipe

Experts (91.7% of params): uniform VQ 3-bit — d=4 subvectors, K=4096 Qwen-native codebook (k-means on group-normalized subvectors), GPTQ error compensation with pooled per-layer input Hessians (ja-centric calibration). 12-bit packed indices + fp16 group scales (eff 3.125 bpw on experts).
Spine: GPTQ 4bit gs64 (linear-attention, attention, shared experts), lm_head 6bit, routers 8bit, embeddings 4bit. Vision tower: unquantized bf16.

Run (Apple Silicon, ≥24 GB unified memory)

bash

pip install "mlx>=0.31" "mlx-lm>=0.31"
hf download aquaman164/Qwen3.6-35B-A3B-MLX-VQ-3.4bpw --local-dir qwen-vq35

python qwen-vq35/code/vq_serve.py --model qwen-vq35 --port 8091   # OpenAI-compatible
python qwen-vq35/code/vq_generate.py qwen-vq35 100                # one-shot test

Decode is the all-12-bit-unpack path: measured 52 tok/s on an M-series 48 GB (vs 66 for the 2.6bpw build — more expert bytes per token).

Provenance / license

Base model © Alibaba Cloud, Apache-2.0. Pipeline: loss-aware allocation + GPTQ-VQ encode (OneCompression); serving adapter = vqmoe model #3.