Qwen/Qwen3.6-35B-A3B quantized with AMD Quark.

Quark's built-in qwen3_5_moe recipe quantizes the routed MoE experts only — 30,720 linears (40 layers x 256 experts x 3 projections). 462 modules are excluded and remain BF16:

lm_head, model.visual.* (vision tower), mtp.*, *mlp.gate, *shared_expert_gate*, *.linear_attn.*, *.self_attn.*, *.shared_expert.*

Because attention, the shared experts, the router and the vision tower all stay BF16, the size reduction is smaller than a whole-model quantization would give.

GSM8K (5-shot, max_gen_toks=2048) and WikiText-2 perplexity, measured with lm-evaluation-harness against the bf16 baseline (Qwen/Qwen3.6-35B-A3B).

Caveat on baseline methodology: the bf16 baselines above were originally distorted by two independent bugs we found and fixed before finalizing these numbers: (1) lm-evaluation-harness's default max_gen_toks=256 truncated this thinking model's <think> traces before reaching an answer (fixed: raised to 2048); (2) a real, reproducible bug in vLLM's Triton implementation of the chunk_gated_delta_rule custom op on this ROCm build causes some bf16 generations to collapse into a fixed degenerate phrase and stop early -- confirmed via a working reference PyTorch implementation (transformers eager, no custom kernel) producing coherent output on the identical prompt/weights, and confirmed in both eager and CUDA-graph-compiled vLLM modes (ruling out graph capture as the cause). Workaround: compilation_config={"custom_ops": ["-chunk_gated_delta_rule"]} forces the correct fallback path. Cross-validated against a 150-example CPU ground-truth run for both base models -- the bug's net effect on the aggregate GSM8K score was small, so the baseline numbers above (measured with the workaround applied) should be reliable. Separately, several checkpoints in this family genuinely score well above the bf16 baseline on GSM8K (a 15-45 point jump in some cases). We verified this is real, not a measurement artifact: it persists identically with and without the kernel-bug workaround applied. It most likely reflects quantization noise suppressing this reasoning model's tendency to enter long, sometimes unproductive chains of self-questioning on the unquantized baseline -- not a genuine capability improvement from quantization. WikiText-2 perplexity (likelihood-based, not generation-based, and unaffected by either bug) shows the expected small regression from quantization and is the more trustworthy signal of quantization quality here.