A training-free coding specialist extracted from Qwen/Qwen3.5-35B-A3B by analytically selecting which mixture-of-experts to keep — no fine-tuning, distillation, or gradient updates of any kind. Half the routed experts are removed; the always-on trunk (attention, shared expert, embeddings) is kept whole.

Parameters: 18.54 B (parent: ~34.6 B) — 128 of 256 routed experts kept per layer
Size: 37.1 GB (bf16, model.safetensors)
Derived from: Qwen3.5-35B-A3B (hybrid linear-attention MoE, top-8 routing)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("deepgoyal19/qwen3.5-35b-a3b-code-extract-N128",
                                         torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("deepgoyal19/qwen3.5-35b-a3b-code-extract-N128")

Recommended coding config (from the parent's card): temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.0, thinking mode on.

Evaluation

Benchmark	pass@1
HumanEval	86.6%
HumanEval+	83.5%

Provenance

census_export.json in this repo records the exact keep-set, N, selector, and parity metrics.