Qwen 3.6 MixedBit Quantized — the Vicegrip of Quantization
Qwen3.6-28B-REAP-mixedbit-tuned-mlx
An MLX "mixedbit" quantization of 0xSero/Qwen3.6-28B (a REAP-pruned Qwen3.5/3.6 hybrid MoE), shipped with the buun hardened chat template and a documented, tested inference config.
The weights are not fine-tuned. This repo's value is: (1) the mixed-bit MLX quant (~13 GB), (2) the bundled hardened chat template, and (3) a battle-tested deployment config — see below.
What is "mixedbit"?
A non-uniform MLX quantization: different layers are stored at different bit-widths instead of a single uniform 4-bit. The result here is ~13 GB on disk — actually smaller than a standard uniform 4-bit quant of the same model (~16–17 GB), because some layers are pushed below 4 bits while sensitive layers are kept higher. It runs well on Apple Silicon via mlx-lm / oMLX.
Architecture: qwen3_5_moe — a hybrid linear-attention + MoE model (40 layers, fused 3-D expert tensors, ~3B active params per token). This matters for tooling (see "Known incompatibilities").
Lineage & credits
Base / weights:0xSero/Qwen3.6-28B — REAP 20%-pruned from Qwen/Qwen3.6-35B-A3B, LoRA-SFT'd on Claude Opus reasoning traces. All model capability is theirs. This repo only re-quantizes those weights to MLX mixedbit.
Chat template:spiritbuun/buun-Qwen3.6-chat_template — hardened Qwen3.5/3.6 template (fixes the official template's "empty-think poisoning" bug). The original stock template is preserved here as chat_template.STOCK.jinja.bak.
Base architecture: Qwen3.5/3.6 by Alibaba/Qwen.
Please honor the licenses/terms of the upstream 0xSero/Qwen3.6-28B and Qwen base models. (No explicit license file was present in the source; defer to upstream.)
Recommended config (tested)
Setting
Value
Why
enable_thinking
False for coding/tool-use/chat
Far more reliable & faster; thinking-ON can ramble.
Thinking budget
~6000 (when thinking ON)
Caps the reasoning loop so it commits instead of spiraling.
Sampling
temp 0.3–0.4, top_p 0.9, top_k 20
(temp ~0.7 for open chat)
fix_mistral_regex
True
Required for this quant's tokenizer.
Two good modes: (1) coding/tool-use → enable_thinking=False (fast, reliable); (2) hard open-ended reasoning → thinking ON + max_tokens ~5000–6000.
⚠️ Known incompatibilities (oMLX speculative features)
Because of the hybrid linear-attention/MoE architecture, these crash and should be left OFF:
DFlash → [convert] Only length-1 arrays can be converted to Python scalars (in target_qwen_gdn). Also single-stream.
SpecPrefill → '_OffsetAdjustedRoPE' object has no attribute 'dims'.
MTP → this quant does not carry the mtp.* tensors.
Plain (non-speculative) decoding works great: ~78 tok/s in chat, ~49 tok/s on large (27K-token) agentic prompts. TurboQuant KV cache @ 6-bit is fine if you want to save runtime memory on long contexts.
Capability (informal, correctly-prompted)
Strong at: math, everyday programming, algorithms (LIS, Dijkstra, DP, thread-safe queues), tool-calling (clean JSON), and agentic planning. Weakness: niche factual recall (e.g. specific geography) — it can hallucinate and ruminate when it lacks knowledge; pair it with a search tool for fact-lookups.
