Gemma 4 A4B 98-Expert v6-coder (C6v3lcb) — LCB-targeted code prune (~20.8B)

✅ Chat template — agentic loop fix (2026-06-10)

Eval complete (Q6_K / llama.cpp). The full canonical 9-bench suite plus the extended LCB-medium-100 and MultiPL-E-100 code benches are filled below, every cell read from summary.json (greedy, cohort-pinned recipe). The 128e and 98e-v5-coder anchor columns are the matching Q6_K reference runs. NVFP4A16 is a deployment format and is not separately benchmarked (cohort policy).

Headline — the LCB targeting worked. LCB-medium-55 96.36% (+10.91pp vs v5-coder, and +1.81pp past the unpruned 128e), closing the −9.10pp hole that motivated the recipe; MultiPL-E macro 88.0% (+7 vs v5-coder, ≈128e); AIME recovers +10pp (53.33 → 63.33). The budget is paid on the non-code axes the LCB-only class weights deprioritized (MATH −3, IFEval −2 vs v5-coder).

Quantized formats

Eval is llama.cpp / Q6_K only for this cohort. NVFP4A16 is published as a vLLM-deployable format but is not benchmarked separately — the Q6_K scoreboard below is representative of the model's quality.

At a glance

Recipe

1. C6 gentle base recipe (unchanged from v5-coder)

strategy      = max              # per-expert score = MAX over classes (not mean/geomean)
normalize     = rank             # rank-normalize within each (layer, class) before aggregating
protect_top   = 16               # the 16 highest-scoring experts/layer are never dropped
alpha         = 2.0              # contribution sharpening exponent
breadth_bonus = 0.5              # reward experts active across many classes (anti-overfit)
v4_floor_map  = v4_layer_floor_map.json   # per-layer floor: never drop below v4's keep on protected layers
baseline      = teacher_force_98e_p16_clean.json   # tie-break anchor

2. v3 calibration data + outlier fix (changed from v5-coder)

data                = expert_neuron_v5_code_v3.json
outlier_mode        = median
outlier_wnorm_thresh = 1e4       # clamp expert weight-norms above 1e4 to the layer median

3. LCB-only targeting — the "lcb" in C6v3lcb (changed)

4. Mandatory shared-FFN α=1.2 (cohort rule)

Expert mapping

• Uniform 30 experts dropped per layer, all 30 layers → 98 kept/layer (128 → 98).
• Mean overlap with the teacher-force baseline: 17.8 / 30 dropped per layer (min 11, max 24). The recipe re-ranks ~12 of 30 dropped slots/layer away from the naive teacher-force drop — that re-ranking is where the code-faithfulness lives.
• Aggregated keep scores (rank-normalized, α=2.0, breadth-bonus) span ~0.0–3.5 per layer with mean ~1.85; the protected top-16 sit at the high end, the dropped 30 at the low end, with boundary_ties_within_1pct 2–39/layer (highest in the shallow layers 0/10 — those layers have the flattest expert-importance profile).

Problems solved

1. Rumination on hard code/reasoning problems. Pruned Gemma 4 variants fall into <think> loops on the hardest LCB / GPQA / AIME problems, blowing the token budget without converging. C6v3lcb attacks this two ways: (a) the LCB-targeted drop map keeps the experts that the hard-problem pass-traces actually use; (b) at eval time the lcb_medium_*_v4 template applies a thinking_token_budget=12288 forcing function (parser=gemma4 + enable_thinking) that caps the rumination loop. T109 rumination-signal gate (GPQA-48 / AIME probes) confirmed the variant generates and converges rather than looping.
2. Data-vs-recipe disentanglement (T102). v6-coder isolates whether v3's earlier regressions were the data or the recipe: holding v5-coder's gentle C6 recipe fixed and only swapping in v3 data + the outlier fix. C6v3 ≈ v5-coder in smoke → the v3 data is fine and C12's aggressive recipe was the rumination cause; C6v3lcb then re-steers that clean baseline at LCB.
3. Deep-layer calibration corruption (T73.0). The _fixed traces were ~86% NaN in deep layers; v6-coder uses real-signal v3 data with a median outlier clamp so deep-layer experts are ranked on genuine contribution, not NaN-blindness.
4. Unfair-comparison trap. The shared α=1.2 step is mandatory and marker-verified, so v6-coder vs v5-coder is apples-to-apples.

Scoreboard — Q6_K GGUF, llama.cpp, greedy

Read: v6-coder lands on/above v5-coder on 8 of 9 benches and beats the unpruned 128e on every code bench (HE +2.44, HE+ +2.44, LCB-55 +1.81). The headline is LCB-medium-55 +10.91pp vs v5-coder — the targeted hole is not just closed but pushed past the base model. AIME recovers +10pp (53.33 → 63.33). The cost lands on MATH (−3) and IFEval (−2) vs v5-coder — the non-code generalist axes the LCB-only class weights (1 1 3 1 1 0 0 2) deliberately deprioritized.

Extended code benches — LCB-medium-100 + MultiPL-E-100 (Q6_K, llama.cpp, greedy)

Coder-field comparison — v6-coder vs Qwen2.5-Coder-14B / 7B + Qwen3.5-9B (Q6_K, llama.cpp, greedy)

• v6-coder — Gemma-4 26B-A4B MoE pruned to 98 experts (~20.8B total, ~A4B active), reasoning.
• Qwen2.5-Coder-14B / 7B-Instruct — dense, non-reasoning code specialists (bartowski Q6_K).
• Qwen3.5-9B — dense reasoning model (bartowski Q6_K).

Note on Qwen3.5-9B. Qwen3.5-9B is a verbose, slow thinking model: it emits long <think> reasoning chains (often ≥1900 tokens even on a trivial GSM8K question), so it runs several× slower per question than the non-reasoning Qwen2.5-Coder models — well beyond what its 9B size would suggest. Its GSM8K / MATH-500 / GPQA cells are being re-run after a harness fix: under batched, reasoning-parsed serving the verbose thinking intermittently left the final answer inside the reasoning block (mis-scored as empty content). ARC (96.76%) and AIME (56.67%) were unaffected.

Answer-length analysis (anti-rumination)

• LCB — v6-coder ruminates less than 128e, under the only definition that matters (long thinking that fails to reach a PASS). The naive read of the median says the opposite — 128e's median LCB completion is ~0.6–1.2k tokens vs v6-coder's budget-capped ~12.7k — but that short median is an artifact of mixing easy and hard problems. 128e only thinks long when it is lost, and then it fails: every 128e output that saturates the budget is a failure (2/2 on LCB-100, 1/1 on LCB-55), and the single longest output across all three models — 51,568 chars / 15,585 tokens on lcb/leetcode/3659 — is a 128e failure, one that both v6-coder (PASS, 12,691 tok / 32,676 chars) and v5-coder solve in two-thirds the length. v6-coder, by contrast, saturates the budget on ~97% of problems but 96% of those long outputs PASS; its longest LCB-55 output is a correct answer and its max length (41–43k chars) sits below 128e's 51k. Wasted long-thinking rate: 128e 100%, v6-coder ~4%. And v6-coder beats 128e on score on both axes (96.36 / 96.00 vs 94.55 / 95.00) — more correct and a lower worst-case tail.
• 128e's routing is erratic in both directions, and the targeted map repairs both. 128e fails LCB two ways: by over-thinking (3659: 51k chars, no answer) and by under-thinking (lcb/leetcode/3000: a premature wrong answer in 579 tokens; 3793: 3101 tok) — on the very same problems v6-coder spends the full ~13k-token budget and passes. This bidirectional instability is the fingerprint of an imperfect base router; the 51k outliers and the lower 128e score are the symptom. v6-coder's LCB-targeted expert map corrects 3 of the 5 problems 128e fails on LCB-100 (2 of 3 on LCB-55). The thinking_token_budget=12288 forcing-function only bounds the loop; the targeted map is what makes the bounded thinking land on a PASS instead of a 51k-char dead end. (Versus v5-coder, v6-coder is statistically the same length — median 12799 vs 12804 tok on LCB-55 — yet passes far more: 53/55 vs 47/55, 96/100 vs 91/100.)
• AIME — genuinely less rumination, +10pp. Median completion 939 tok vs v5-coder's 1156, p90 char tail 8404 < 9239, scoring 63.33% vs 53.33% — shorter chains and more correct.
• GSM8K — tightest of the three (max 396 tok; 128e 1107, v5-coder 858); no rumination tail.
• IFEval — no ruminating outlier (max 7200 chars vs v5-coder's 17539; ≈128e 7425).
• GPQA — a base-model ceiling, not a prune artifact. p90 completion tokens peg at ~8189 on all three models; the hard-question rumination is inherited from Gemma 4 and is identical across the cohort, not widened by pruning.
• ARC — one shared single-sample outlier (~16266 tok / ~44k chars) on both pruned variants, absent on 128e; p50/p90 normal and identical, so not systemic length growth.
• MultiPL-E — most compact code of the three. On the bench where the answer is the code (extracted final block, no reasoning trace), v6-coder has the tightest tail — max 2169 chars vs 128e 2944 / v5-coder 3083, p90 591 chars / 184 tok — at ≈128e pass-rate (macro 88.0%).
• Net: v6-coder is more accurate than 128e on every LCB axis (96.36 / 96.00 / MultiPL-E macro 88.0) at a higher average inference cost (it thinks the full bounded budget) but a lower worst case (max 41–43k chars vs 128e's 51k) and a far lower wasted-thinking rate (~4% vs 100%). On the short-reasoning benches (AIME / GSM8K / IFEval) it additionally reduces length vs v5-coder. The targeted map never traded length for accuracy — it traded 128e's erratic, sometimes-catastrophic tail for a higher but bounded and productive average.

Methodology / data integrity. Per-problem lengths come from omk_eval token_stats over each bench's per-problem samples archive — lm-eval samples_*.jsonl for the 9-bench, and the native runners' lcb_result.samples.jsonl / mpe_result.samples.jsonl (mirrored into the durable sqlite resume cache) for LCB/MPE. MultiPL-E is tabulated, but its rows measure extracted-code length, not reasoning: its samples store only the final code block (no <think> trace, finish_reasons all terminal, thinking_tokens_est=0), so MPE is a code-conciseness reference — not a rumination metric — on which v6-coder is the most compact. The reasoning-bearing benches (LCB / GPQA / AIME / MATH) carry the rumination signal; MPE tokens there are tokenizer-estimated (the code-extraction path returns no server token count). thinking_tokens_est is 0 across all benches under llama.cpp --reasoning-format deepseek (reasoning is emitted inline, not as a separable block), so the completion-token columns are the length signal. The LCB/MPE token_stats are computed over per-problem rows keyed on doc_id (the native runners now emit it, fixed 2026-05-25) — earlier card revisions briefly collapsed these to n=1; all numbers here are at full n (LCB-55 n=55, LCB-100 n=100, MultiPL-E n=300).

Reasoning budget and thinking stop phrase (llama.cpp)

Serve with a bounded thinking block

Two wordings that work

What it measures out to

Per request, instead of per server

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Reproduce

Lineage & provenance

• Base: google/gemma-4-26B-A4B-it (128e, fresh prune — not derived from v4/v5 weights).
• Cohort: v5-coder (C6 _fixed) → C6v3 (C6 v3-data) → C6v3lcb (this, LCB-targeted).
• Tooling: omnimergekit — expert_drop.py, generate_drop_map_multiclass.py, router_shared_upweight.py; eval via omk_eval.py + EVAL_PROTOCOL.
• Eval sampler is frozen GREEDY across the whole cohort for apples-to-apples comparison.

• Gemma License (inherits from Google Gemma 4)

• Apache-2.0