A 24 GiB mixed-precision MLX quantization of
Qwen/Qwen3.6-35B-A3B —
built for 32 GB Macs, and the first MLX build of this project to beat
the source Dynamic GGUF's imatrix rounding at short context.
Native MLX, no strings attached. Standard MLX affine layout —
bit-identical packing to mx.quantize, no custom kernels, no forked
runtime. Loads in anything that speaks MLX (mlx-vlm, mlx-lm-based
servers, LM Studio) at exactly the speed of any stock-converted model.
The vision tower is fully functional.
Method (brief)
Winter-Mix: sensitivity-informed mixed-precision allocation with
hard pins on routing-critical tensors, calibrated error-compensated
rounding (GPTQ-family, independent MLX-native implementation), and a
diverse long-context calibration mixture engineered for full expert
coverage — including multilingual content so language-specialist
experts are calibrated rather than silently degraded. Validated against
RTN controls, an 8-bit reference, and cross-engine baselines. The full
pipeline is not released.
Measured quality
One scoring rule for every row — NLL over the second half of each
context window, token-aligned across engines (llama.cpp's native rule).
Comparison rows are benchmarked reference points from the same harness,
not published artifacts.
model
GiB
short-2K ppl
long-16K ppl
8-bit MLX reference
35
6.0275
4.6008
WinterMix58
24
6.0420
4.6167
Unsloth UD-Q5_K_XL GGUF (llama.cpp)
24.8
6.0504
4.6060
oQ5 (oMLX)
24
6.0806
4.6350
oQ4 (oMLX)
20
6.1221
4.6934
Beats the source GGUF at short context (−0.14%) at a slightly
smaller size; the GGUF keeps a small 16K edge (+0.23%).
Beats its exact size-peer oQ5 in both bands (−0.63% short,
−0.39% long).
Sits just +0.24% / +0.35% behind the full 8-bit reference — at
11 GiB less.
Long-context checked to 98K tokens with no depth collapse;
multilingual (8-script) parity verified.
Beyond perplexity
Selected using three instruments — NLL, seeded behavioral probes, and
direct reading of reasoning traces. On a 30-step state-tracking
benchmark with knife-edge conditional rules (5 seeded runs, 50K-token
context), WinterMix58 delivered a correct final state in 5 of 5
runs — the only build in the evaluation family to do so, including the
8-bit reference — with zero fabricated figures and zero truncations.
Its reasoning traces show scheduled verification passes that resolve
forward and catch-and-correct behavior before delivery. Measured
generation speed during these runs: ~70 tok/s at 50K context on an
M5 Max.
Running on 32 GB Macs
The model occupies 24 GiB; this architecture's KV cache is unusually
light (~2 GB per 100K-token session — only 10 of 40 layers are full
attention). On a 32 GB machine, raise the GPU wired limit, e.g.:
sudo sysctl iogpu.wired_limit_mb=28672
leaving room for a 100K+ context session. On 48 GB+ machines it runs
with no configuration at all.
Provenance & credits
Base model: Qwen/Qwen3.6-35B-A3B
— quantized directly from the BF16 release; Apache 2.0 inherited.
Bit allocation informed by analysis of the tensor-type metadata in
Unsloth's published Dynamic GGUF
(metadata analysis only; no weights or code reused).
