v2 is a two-stage imatrix → DWQ build (reasoning, then math/code) of the same base.
It is materially better on code — HumanEval 90.2 % (this v1 line: 84.8 %) ·
LiveCodeBench 39.9 % (v1: 32 %) · MBPP 75.4 % — at the same size class, same
MTP packaging.
This v1 repo is kept for reproducibility only. New users should download v2.
This is a text build of the reasoning-distilled
Qwen3.6-35B-A3B, packaged for fast local inference on Apple Silicon with an embedded
Multi-Token-Prediction (MTP) head for speculative decoding.
⚠️ These are quantized weights. All capability comes from the base modellordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — please star/cite it first. This repo's
contribution is the quantization + MTP packaging and its fidelity/speed validation.
Model lineage
Qwen/Qwen3.6-35B-A3B (Apache-2.0 base, 35B MoE · 256 experts · ~3B active · 262k ctx)
└─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled (SFT distillation of Claude Opus 4.7 reasoning)
└─ THIS REPO: DWQ 4-bit MLX + MTP
Architecture: Qwen3.6-35B-A3B — Mixture-of-Experts, 256 experts (8 routed + 1 shared),
~3B active parameters/token, 40 layers (linear + full attention), up to 262,144-token
context (the base card recommends ≤64k for best quality).
Reasoning distillation (by lordx64): SFT on ~7,800
chain-of-thought traces from Claude Opus 4.7 (Anthropic), teaching explicit
<think>…</think> reasoning. Attention-only LoRA (r=16) via Unsloth + TRL.
This repo: 4-bit MLX quantization (DWQ) with MTP preserved.
This is a DWQ quant — Apple MLX's learned quantization
(mlx-lm LEARNED_QUANTS).
A uniform 4-bit student is gradient-distilled to match an 8-bit teacher of the same
model (KL divergence on the top-1024 logits, temperature 2.0, lr 1e-6), so the flat 4-bit
weights recover most of the fidelity that plain round-to-nearest 4-bit loses.
Scheme
affine uniform 4-bit, group size 64 (router gate / shared_expert_gate at 8-bit, the mlx-lm default)
Effective size
~4.3 bits/weight — uniform, so fast dense matmuls
Distillation teacher
8-bit MLX quant of the same base
Calibration
2,856 pre-split ≤512-token windows (reasoning tails covered, not just heads)
Result
validation KL-to-teacher 0.0334 → 0.0325 (improved) over the undistilled 4-bit start
Tooling
mlx_lm.quant.dwq (MLX 0.31.x)
Scope: this is the DWQ stage of MLX's Dynamic → DWQ recipe. The student is a
uniform 4-bit convert (round-to-nearest), not a Dynamic-quant mixed-precision student —
so fidelity is recovered by distillation, not by per-layer bit allocation. The sibling
oQ4 repos are the sensitivity-mixed (Dynamic-style) quant on their own; a full
Dynamic → DWQ build (mixed precision then distillation) is a natural next step.
Because DWQ produces a uniform 4-bit model, it is both smaller and slightly faster
than sensitivity-mixed schemes at equal task quality (see Evaluation).
Evaluation
Two kinds of numbers below: quant-fidelity metrics measured on this quant, and
capability benchmarks inherited from the bf16 base (a 4-bit quant preserves — does not
create — capability; the fidelity metrics show how little was lost).
Quant fidelity (measured on this model, served via oMLX)
Metric
This model
Notes
Gradeable math (10 short-answer problems)
10 / 10
arithmetic, GCD, algebra, calculus, combinatorics — all complete within budget
Note on long-form math. This is a reasoning model that routinely emits 8,000+ tokens
of <think> on olympiad problems. Fixed-budget pass/fail on such problems is
token-budget-limited, not quantization-limited — every 4-bit and 8-bit level behaves
the same. Give it a large max_tokens (32k) for hard problems.
Inherited base-model benchmarks (bf16, from the base card)
Reported by the base model lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled via lm-evaluation-harness; listed here as a
capability reference for the family, not re-measured on this quant.
Benchmark
Base (bf16)
GSM8K (8-shot CoT)
84.3 %
MMLU-Pro (5-shot)
74.9 %
MMLU-Pro · Math
83.6 %
MMLU-Pro · Physics
81.0 %
MMLU-Pro · Computer Science
79.0 %
Sibling repos (same base, pick your trade-off)
Repo
Quant
Vision
Size
Best for
…-DWQ4-MTP-Vision-MLX
DWQ uniform 4-bit
✓
~21 GB
smallest+fastest, multimodal
…-DWQ4-MTP-MLX
DWQ uniform 4-bit
–
~19 GB
smallest+fastest, text only
…-oQ4-MTP-Vision-MLX
oq mixed 4–8-bit
✓
~22 GB
mixed-precision, multimodal
…-oQ4-MTP-MLX
oq mixed 4–8-bit
–
~21 GB
mixed-precision, text only
TL;DR: the DWQ builds are smaller, a touch faster, and match the oQ builds on task
quality — distillation lets uniform 4-bit keep up with mixed precision. oQ builds spend
extra bits on sensitive layers (a more conservative, classic dynamic quant).
How to run
These are MLX weights (Apple Silicon). The tested serving path is oMLX,
which supports this model's native MTP speculative decoding out of the box.
Text-only (no vision tower) — leanest footprint for pure text + MTP.
1. Place the folder in your oMLX models directory:
bash
1# after downloading this repo:2mv Qwen3.6-35B-A3B-Claude-4.7-Opus-DWQ4-MTP-MLX ~/.omlx/models/
2. Enable MTP for this model (one-time), then call the OpenAI-compatible API:
bash
1curl -X PUT http://127.0.0.1:8003/admin/api/models/qwen3.6-35b-a3b-claude-4.7-opus-dwq4-mtp-mlx/settings \2 -H "Authorization: Bearer sk-local" -H "Content-Type: application/json"\3 -d '{"mtp_enabled": true}'45curl -X POST http://127.0.0.1:8003/v1/chat/completions \6 -H "Authorization: Bearer sk-local" -H "Content-Type: application/json"\7 -d '{"model": "qwen3.6-35b-a3b-claude-4.7-opus-dwq4-mtp-mlx",
8 "messages": [{"role": "user", "content": "Prove there are infinitely many primes."}],
9 "max_tokens": 8000, "temperature": 0.6}'
The uniform 4-bit backbone also loads directly in stock mlx-lm as a qwen3_5_moe text model (verified).
MTP speculative decoding needs an MTP-aware runtime — oMLX (tested), or mlx-lm with PR #990.
Recommended sampling
temperature 0.6, top_p 0.95, top_k 20. For hard reasoning set max_tokens ≥ 32000 — the
model thinks in explicit <think>…</think> blocks and can be long.
Intended use & limitations
Built for hard reasoning: competition math, STEM, code walk-throughs, multi-step logic.
It thinks in long <think> blocks — budget tokens accordingly.
Reasoning ≠ knowledge. Quantization (and the base distillation) transfer how to reason,
not new facts.
Quantization loss: 4-bit is lossy vs bf16. The fidelity metrics above quantify it
(small); for maximum quality use the base bf16 or an 8-bit quant.
Distillation provenance: the base's training traces were generated with Anthropic's
Claude Opus 4.7. Downstream users should confirm compliance with
Anthropic's usage policy.
Quant calibration (DWQ): the model's own SFT distribution (the dataset above) plus supplementary chain-of-thought reasoning traces, cut into 2,856 ≤512-token windows to cover deep reasoning tails. No new knowledge is introduced — calibration only aligns the 4-bit scales to the model's own outputs.
Acknowledgements
lordx64 — the reasoning-distilled base model this repo quantizes. All capability is theirs.
Qwen team — Qwen3.6-35B-A3B, released under Apache-2.0.
Anthropic — Claude Opus 4.7, the reasoning teacher for the base distillation.
Apple MLX — mlx, mlx-lm (DWQ · mlx_lm.quant.dwq), mlx-vlm.
oMLX — the oq quantizer, MTP serving runtime, and OpenAI-compatible API.
Unsloth & EleutherAI lm-evaluation-harness — used by the base model (training / eval).
License
Apache-2.0, inherited from the base model and Qwen3.6-35B-A3B.
Citation
bibtex
1@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
2 howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
3@misc{lordx64_qwen36_distill_2026,
4 title={Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning}, author={lordx64}, year={2026},
5 howpublished={\url{https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled}} }
