Text-only. Vision-tower weights are not present in this quantization (the base model's vision encoder was excluded during conversion). Despite config.json still declaring Qwen3_5MoeForConditionalGeneration (required for mlx-lm compatibility), image inputs will fail. For multimodal use, see the original Qwen/Qwen3.6-35B-A3B.

~20 GB on disk. Runs on Apple Silicon Macs with 64 GB unified memory for comfortable generation.

Lineage

Qwen/Qwen3.6-35B-A3B
  └─ tvall43/Qwen3.6-35B-A3B-heretic   (abliterated)
       └─ cspenn/Qwen3.6-35B-A3B-heretic-MLX-Mixed-4-8   (this repo, MLX quant)

Qwen/Qwen3.6-35B-A3B — Qwen's open-weight 35B MoE model with only 3B parameters activated per token. Features 40 hybrid transformer layers (alternating Gated DeltaNet linear-attention and Gated Attention blocks), 256 routed experts + 1 shared expert, and a native context window of 262,144 tokens. Notable for strong agentic/coding performance and multimodal capability.

tvall43/Qwen3.6-35B-A3B-heretic — A decensored derivative produced with the Heretic v1.2.0 abliteration technique. Safety refusals were reduced from ~86/100 to ~5/100 by modifying attention output projection weights (attn.o_proj) on a per-layer basis, while maintaining a KL divergence of just 0.0097 from the original model — preserving general behavior almost entirely.

cspenn/Qwen3.6-35B-A3B-heretic-MLX-Mixed-4-8 (this repo) — The heretic weights quantized for Apple Silicon using mlx-lm's convert with a custom quant_predicate. A sensitivity-informed mixed strategy keeps routing and boundary layers at 8-bit while compressing the bulk expert weights to 4-bit. See Quantization Layout below.

Model Details

Property	Value
Base architecture	`Qwen3_5MoeForConditionalGeneration`
Total parameters	~35B
Active parameters per token	~3B
Transformer layers	40 hybrid (30 Gated DeltaNet linear-attn + 10 Gated self-attn)
Self-attn layer positions	3, 7, 11, 15, 19, 23, 27, 31, 35, 39
Routed experts	256
Shared experts	1
Experts activated per token	8 routed + 1 shared
Context length (base)	262,144 tokens
Base dtype	bfloat16
Vision tower	Not present (excluded at quantization time)
Quantized format	MLX affine quantization (safetensors)
Shards	4 safetensors files
On-disk size	~20 GB

Quantization Layout

Default: 4-bit affine, group_size=64. Explicit 8-bit overrides are applied to 132 tensors across six categories based on sensitivity analysis and layer-position anchoring.

Tensor pattern	Scope	Tensor count	Bits	Rationale
`embed_tokens`	input embedding	1	8	Input boundary; errors amplify through all layers
`lm_head`	output head	1	8	Output head sensitivity at the prediction boundary
`mlp.gate`	MoE router in all 40 layers	40	8	Expert routing decisions are disproportionately sensitive to precision
`shared_expert_gate`	shared-expert gate in all 40 layers	40	8	Gates token flow to the always-active shared expert
`linear_attn.out_proj`	all 30 linear-attn (Gated DeltaNet) layers	30	8	Identified by OptiQ sensitivity analysis as the highest-KLD tensor (KLD ~6.0) in these blocks
All tensors in `layers.0.*`	first transformer block	10	8	Input anchor — first and last blocks are known to be especially sensitive
All tensors in `layers.39.*`	last transformer block	10	8	Output anchor
Expert FFN (`switch_mlp.gate_proj`, `switch_mlp.up_proj`, `switch_mlp.down_proj`), shared-expert FFN, attention projections in layers 1–38	bulk of the model	~380	4	Holds the vast majority of ~35B parameters; quality loss is acceptable at 4-bit given MoE sparsity
Vision tower (`.visual.`)	—	0	—	Excluded entirely — small relative to LM and not needed for text-only use

Design rationale: MoE expert weights tolerate 4-bit compression well because only 8 of 256 experts activate per token, limiting quantization noise accumulation. Routing gates, embeddings, and the most sensitive linear-attention projection are kept at 8-bit to preserve routing quality and reduce perplexity degradation. The first and last transformer blocks are anchored at 8-bit as an additional safety margin. All tensors use group_size=64.

Usage

mlx-lm (Python)

python

from mlx_lm import load, generate

model, tokenizer = load("cspenn/Qwen3.6-35B-A3B-heretic-MLX-Mixed-4-8")

messages = [{"role": "user", "content": "Explain mixture-of-experts in one paragraph."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)

mlx-lm (CLI)

bash

mlx_lm.generate \
  --model cspenn/Qwen3.6-35B-A3B-heretic-MLX-Mixed-4-8 \
  --prompt "Explain mixture-of-experts in one paragraph." \
  --max-tokens 512

LM Studio

LM Studio supports MLX safetensors natively on Apple Silicon. Either:

From HuggingFace: Search for cspenn/Qwen3.6-35B-A3B-heretic-MLX-Mixed-4-8 in the Discover tab.
Local folder: Use Load Model from Disk and point to the downloaded folder. The chat_template.jinja and tokenizer.json are included, so chat templating works out of the box.

Important Caveats

Text-only. config.json retains Qwen3_5MoeForConditionalGeneration (a VLM architecture class) and the image_token_id field because altering these would break mlx-lm's model loader. However, the vision encoder weights were not included in the safetensors; any attempt to pass image inputs will raise an error. This is intentional.
Heretic variant. This model is derived from an abliterated base. It is uncensored and will comply with requests the original Qwen model would refuse. Use responsibly.
Memory. ~64 GB unified memory is recommended for comfortable inference. The model loads in ~20 GB of weight but requires additional memory for the KV cache at long context lengths.
Thinking mode. Qwen3.6 supports an explicit <think> reasoning mode. The heretic abliteration was applied without disabling this capability; it should remain functional.

License

Apache 2.0 — same as the upstream Qwen model. See LICENSE.

This derivative is distributed under the same terms. Commercial use is permitted.

Credits

Qwen team (Alibaba) — original Qwen3.6-35B-A3B architecture and weights.
tvall43 — Heretic v1.2.0 abliteration producing the decensored base.
mlx-lm — Apple's MLX framework and convert utility used for quantization.
Quantization — produced by cspenn using a custom quant_predicate informed by OptiQ sensitivity analysis.