gemma-4-A4B-98e-v6-coder-it-GGUF

✅ Chat template — agentic loop fix (2026-06-10)

The anti-repetition sampler flags (--min-p 0.05 --repeat-penalty 1.1) noted under How to Use are no longer required to prevent this agentic loop; they remain safe as general sampling. --reasoning-budget 8192 is still required for Gemma 4 thinking.

Available Quantizations

imatrix policy. The plain K-quant ladder was rebuilt with imatrix (calibration data v5) on 2026-05-25, which lifted the low-bit K-quants the most (e.g. Q3_K_M 85.98 → 92.68, Q3_K_S 82.93 → 87.80, Q3_K_L/XL 88.41 → 92.07). Q4_K_M is the one exception — built without imatrix, because imatrix measurably lowered its HE+ (90.85 imatrix vs 92.07 plain). IQ-tiers always carry imatrix; Q8_0/Q4_0/Q4_1 never do.

Recommended: IQ4_XS (11.01 GB / 92.07%) is the safe 4-bit default. The imatrix rebuild opened a 92.68% cluster just below it — Q3_K_M (10.51 GB), Q4_K_S (12.21 GB), Q4_K_L (13.42 GB), Q5_K_M (15.07 GB) — so Q3_K_M is now the smallest tier at the top of the HE+ band (the ≤0.61pp steps here are within HE+'s 1-question granularity, so treat the 92.07/92.68 cluster as a tie and pick by disk). v6-coder holds ≥92.07% down to IQ3_XS (9.22 GB / 3.54 bpw). Max fidelity: Q6_K / Q8_0 (93.29%). Avoid IQ2_XS (70.73% — 2-bit cliff; use IQ2_S 89.02% at 7.83 GB instead).

How to Use

Reasoning budget and thinking stop phrase (llama.cpp)

Serve with a bounded thinking block

Two wordings that work

What it measures out to

Per request, instead of per server

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Comparison vs the 14–22B coder field

• HE / HE+: top of the band — 98.78 / 93.29, ~+9 / +6pp above Qwen2.5-Coder-14B's 89.6 / 87.2 (the published field leader). Essentially tied with v5-coder (−0.61 HE, identical HE+) while gaining +10.91pp on LCB.
• LCB: v6-coder's 96.36 is LCB-medium-55 on v4 problems. The published Qwen2.5-Coder / DS-Coder-V2-Lite numbers above (23.4 / 24.3) are full LCB on pre-v4 windows — a different, non-comparable subset. The apples-to-apples number now exists: run on the same LCB-medium-55 v4 split and the same llama.cpp stack, Qwen2.5-Coder-14B scores 18.18% vs v6-coder's 96.36%. Read it as a same-rig measurement, not a definitive ranking — the v4 split is only 55 problems, v6-coder was specifically LCB-v4-targeted (that is the entire recipe), and Qwen takes the same llama-server chat-template penalty seen in the same-stack HE+ sweep below. Don't read the raw +78pp as general coding superiority.
• MATH-500 91.00 / GSM8K 91.00 / AIME 63.33: top of the band for math-on-text reasoning. Phi-4's 80.4 MATH is the closest generalist; v6-coder beats it by ~11pp. AIME 63.33 is far above any published 14–22B coder AIME.
• GPQA-Diamond 67.17 / IFEval 92.00: GPQA materially above Phi-4 (56.1) and Qwen2.5-14B (40.9). IFEval 92 beats Phi-4 (63.0) and Qwen2.5-14B (80.0).

v6-coder Q4_K_M vs Qwen3.6-35B-A3B-UD-IQ3_XXS — iso-disk 10-bench head-to-head

• Code (the v6-coder remit) — v6 wins all four. MultiPL-E +18.00pp (88.33 vs 70.33), LCB-medium-55 +5.46, HE+ +3.05, HE +1.83. The smaller 20.8B code-pruned MoE beats the 35B generalist on every execution-scored code bench, at lower disk and on its weakest (no-imatrix) 4-bit tier.
• Math / science — Qwen wins. GPQA-D +11.61, AIME +13.33, gsm8k / math500 +6 / +9 (on the corrected templates). A full-breadth 35B generalist out-reasons the code-specialized prune on math-on-text and graduate science — expected, and the gap is the cost of the specialization.
• General — a wash. IFEval (92 vs 93) and ARC-Challenge (96.08 vs 96.84) are within a point.

Same-Stack GGUF HE+ Sweep — v6-coder vs Qwen2.5-Coder-14B-Instruct

Qwen2.5-Coder-14B-Instruct (14.7B dense) — bartowski quants

Head-to-head by file size (v6-coder runs lower bpw at the same disk)

1. A 92.68% cluster over a wide 92.07% plateau. The imatrix rebuild lifted the K-quants into a 92.68% top cluster — Q3_K_M (10.51 GB), Q4_K_S (12.21 GB), Q4_K_L (13.42 GB), Q5_K_M (15.07 GB) — sitting just above a broad 92.07% plateau (IQ3_XS 9.22 GB up through IQ4_XS / Q4_K_M / Q5_K_S / IQ3_M / Q3_K_L / Q3_K_XL). The 0.61pp steps are within HE+'s 1-question granularity, so pick by disk: Q3_K_M (10.51 GB) is the smallest tier in the top cluster, IQ3_XS (9.22 GB) the smallest on the plateau, IQ4_XS (11.01 GB) the safe 4-bit default.
2. vs Qwen, every band wins by +4 to +9pp at 1.5–3.5 bpw less. The biggest gap is the iso-disk ~10.5 GB point — v6 Q3_K_M (10.51 GB / 4.04 bpw / 92.68%) vs Qwen Q5_K_M (10.51 GB / 5.72 bpw / 83.54%): +9.14pp at the exact same file size, −1.68 bpw.
3. Sub-8 GB code-grade. IQ2_S (7.83 GB / 3.01 bpw / 89.02%) beats Qwen's IQ4_XS (8.12 GB / 4.42 bpw / 84.76%) by +4.26pp at smaller disk and lower bpw. IQ2_XS (7.77 GB) is the 2-bit cliff at 70.73% — use IQ2_S.
4. Sweep complete; imatrix lifted the low-bit K-quants most. Q8_0 / Q6_K top the cohort at 93.29%; the imatrix rebuild moved the 3-bit K-quants up sharply (Q3_K_M 85.98 → 92.68, Q3_K_S 82.93 → 87.80, Q3_K_L/XL 88.41 → 92.07), so a 3-bit tier (Q3_K_M, 10.51 GB) now leads the value frontier. The 4 CD-* contrib-dynamic tiers fill the 10–15.5 GB bands (CD-Q6_K 92.07, CD-Q4_K_M / CD-IQ4_K_M 91.46, CD-Q5_K_M 90.85). Below 9 GB the plain IQ ladder leads (IQ3_XS 9.22 GB / 92.07%, IQ2_S 7.83 GB / 89.02%).

CD recipes are open-source — generator at omnimergekit/scripts/generate_cd_maps.py.

Original Model Card

Gemma 4 A4B 98-Expert v6-coder (C6v3lcb) — LCB-targeted code prune (~20.8B)

Eval complete (Q6_K / llama.cpp). The full canonical 9-bench suite plus the extended LCB-medium-100 and MultiPL-E-100 code benches are filled below, every cell read from summary.json (greedy, cohort-pinned recipe). The 128e and 98e-v5-coder anchor columns are the matching Q6_K reference runs. NVFP4A16 is a deployment format and is not separately benchmarked (cohort policy).

Headline — the LCB targeting worked. LCB-medium-55 96.36% (+10.91pp vs v5-coder, and +1.81pp past the unpruned 128e), closing the −9.10pp hole that motivated the recipe; MultiPL-E macro 88.0% (+7 vs v5-coder, ≈128e); AIME recovers +10pp (53.33 → 63.33). The budget is paid on the non-code axes the LCB-only class weights deprioritized (MATH −3, IFEval −2 vs v5-coder).

Quantized formats

Eval is llama.cpp / Q6_K only for this cohort. NVFP4A16 is published as a vLLM-deployable format but is not benchmarked separately — the Q6_K scoreboard below is representative of the model's quality.

At a glance

Recipe

1. C6 gentle base recipe (unchanged from v5-coder)

strategy      = max              # per-expert score = MAX over classes (not mean/geomean)
normalize     = rank             # rank-normalize within each (layer, class) before aggregating
protect_top   = 16               # the 16 highest-scoring experts/layer are never dropped
alpha         = 2.0              # contribution sharpening exponent
breadth_bonus = 0.5              # reward experts active across many classes (anti-overfit)
v4_floor_map  = v4_layer_floor_map.json   # per-layer floor: never drop below v4's keep on protected layers
baseline      = teacher_force_98e_p16_clean.json   # tie-break anchor

2. v3 calibration data + outlier fix (changed from v5-coder)

data                = expert_neuron_v5_code_v3.json
outlier_mode        = median
outlier_wnorm_thresh = 1e4       # clamp expert weight-norms above 1e4 to the layer median

3. LCB-only targeting — the "lcb" in C6v3lcb (changed)

4. Mandatory shared-FFN α=1.2 (cohort rule)

Expert mapping

• Uniform 30 experts dropped per layer, all 30 layers → 98 kept/layer (128 → 98).
• Mean overlap with the teacher-force baseline: 17.8 / 30 dropped per layer (min 11, max 24). The recipe re-ranks ~12 of 30 dropped slots/layer away from the naive teacher-force drop — that re-ranking is where the code-faithfulness lives.
• Aggregated keep scores (rank-normalized, α=2.0, breadth-bonus) span ~0.0–3.5 per layer with mean ~1.85; the protected top-16 sit at the high end, the dropped 30 at the low end, with boundary_ties_within_1pct 2–39/layer (highest in the shallow layers 0/10 — those layers have the flattest expert-importance profile).

Problems solved

1. Rumination on hard code/reasoning problems. Pruned Gemma 4 variants fall into <think> loops on the hardest LCB / GPQA / AIME problems, blowing the token budget without converging. C6v3lcb attacks this two ways: (a) the LCB-targeted drop map keeps the experts that the hard-problem pass-traces actually use; (b) at eval time the lcb_medium_*_v4 template applies a thinking_token_budget=12288 forcing function (parser=gemma4 + enable_thinking) that caps the rumination loop. T109 rumination-signal gate (GPQA-48 / AIME probes) confirmed the variant generates and converges rather than looping.
2. Data-vs-recipe disentanglement (T102). v6-coder isolates whether v3's earlier regressions were the data or the recipe: holding v5-coder's gentle C6 recipe fixed and only swapping in v3 data + the outlier fix. C6v3 ≈ v5-coder in smoke → the v3 data is fine and C12's aggressive recipe was the rumination cause; C6v3lcb then re-steers that clean baseline at LCB.
3. Deep-layer calibration corruption (T73.0). The _fixed traces were ~86% NaN in deep layers; v6-coder uses real-signal v3 data with a median outlier clamp so deep-layer experts are ranked on genuine contribution, not NaN-blindness.
4. Unfair-comparison trap. The shared α=1.2 step is mandatory and marker-verified, so v6-coder vs v5-coder is apples-to-apples.

Scoreboard — Q6_K GGUF, llama.cpp, greedy

Read: v6-coder lands on/above v5-coder on 8 of 9 benches and beats the unpruned 128e on every code bench (HE +2.44, HE+ +2.44, LCB-55 +1.81). The headline is LCB-medium-55 +10.91pp vs v5-coder — the targeted hole is not just closed but pushed past the base model. AIME recovers +10pp (53.33 → 63.33). The cost lands on MATH (−3) and IFEval (−2) vs v5-coder — the non-code generalist axes the LCB-only class weights (1 1 3 1 1 0 0 2) deliberately deprioritized.

Extended code benches — LCB-medium-100 + MultiPL-E-100 (Q6_K, llama.cpp, greedy)

Coder-field comparison — v6-coder vs Qwen2.5-Coder-14B / 7B + Qwen3.5-9B (Q6_K, llama.cpp, greedy)

• v6-coder — Gemma-4 26B-A4B MoE pruned to 98 experts (~20.8B total, ~A4B active), reasoning.
• Qwen2.5-Coder-14B / 7B-Instruct — dense, non-reasoning code specialists (bartowski Q6_K).
• Qwen3.5-9B — dense reasoning model (bartowski Q6_K).

Note on Qwen3.5-9B. Qwen3.5-9B is a verbose, slow thinking model: it emits long <think> reasoning chains (often ≥1900 tokens even on a trivial GSM8K question), so it runs several× slower per question than the non-reasoning Qwen2.5-Coder models — well beyond what its 9B size would suggest. Its GSM8K / MATH-500 / GPQA cells are being re-run after a harness fix: under batched, reasoning-parsed serving the verbose thinking intermittently left the final answer inside the reasoning block (mis-scored as empty content). ARC (96.76%) and AIME (56.67%) were unaffected.

Answer-length analysis (anti-rumination)

• LCB — v6-coder ruminates less than 128e, under the only definition that matters (long thinking that fails to reach a PASS). The naive read of the median says the opposite — 128e's median LCB completion is ~0.6–1.2k tokens vs v6-coder's budget-capped ~12.7k — but that short median is an artifact of mixing easy and hard problems. 128e only thinks long when it is lost, and then it fails: every 128e output that saturates the budget is a failure (2/2 on LCB-100, 1/1 on LCB-55), and the single longest output across all three models — 51,568 chars / 15,585 tokens on lcb/leetcode/3659 — is a 128e failure, one that both v6-coder (PASS, 12,691 tok / 32,676 chars) and v5-coder solve in two-thirds the length. v6-coder, by contrast, saturates the budget on ~97% of problems but 96% of those long outputs PASS; its longest LCB-55 output is a correct answer and its max length (41–43k chars) sits below 128e's 51k. Wasted long-thinking rate: 128e 100%, v6-coder ~4%. And v6-coder beats 128e on score on both axes (96.36 / 96.00 vs 94.55 / 95.00) — more correct and a lower worst-case tail.
• 128e's routing is erratic in both directions, and the targeted map repairs both. 128e fails LCB two ways: by over-thinking (3659: 51k chars, no answer) and by under-thinking (lcb/leetcode/3000: a premature wrong answer in 579 tokens; 3793: 3101 tok) — on the very same problems v6-coder spends the full ~13k-token budget and passes. This bidirectional instability is the fingerprint of an imperfect base router; the 51k outliers and the lower 128e score are the symptom. v6-coder's LCB-targeted expert map corrects 3 of the 5 problems 128e fails on LCB-100 (2 of 3 on LCB-55). The thinking_token_budget=12288 forcing-function only bounds the loop; the targeted map is what makes the bounded thinking land on a PASS instead of a 51k-char dead end. (Versus v5-coder, v6-coder is statistically the same length — median 12799 vs 12804 tok on LCB-55 — yet passes far more: 53/55 vs 47/55, 96/100 vs 91/100.)
• AIME — genuinely less rumination, +10pp. Median completion 939 tok vs v5-coder's 1156, p90 char tail 8404 < 9239, scoring 63.33% vs 53.33% — shorter chains and more correct.
• GSM8K — tightest of the three (max 396 tok; 128e 1107, v5-coder 858); no rumination tail.
• IFEval — no ruminating outlier (max 7200 chars vs v5-coder's 17539; ≈128e 7425).
• GPQA — a base-model ceiling, not a prune artifact. p90 completion tokens peg at ~8189 on all three models; the hard-question rumination is inherited from Gemma 4 and is identical across the cohort, not widened by pruning.
• ARC — one shared single-sample outlier (~16266 tok / ~44k chars) on both pruned variants, absent on 128e; p50/p90 normal and identical, so not systemic length growth.
• MultiPL-E — most compact code of the three. On the bench where the answer is the code (extracted final block, no reasoning trace), v6-coder has the tightest tail — max 2169 chars vs 128e 2944 / v5-coder 3083, p90 591 chars / 184 tok — at ≈128e pass-rate (macro 88.0%).
• Net: v6-coder is more accurate than 128e on every LCB axis (96.36 / 96.00 / MultiPL-E macro 88.0) at a higher average inference cost (it thinks the full bounded budget) but a lower worst case (max 41–43k chars vs 128e's 51k) and a far lower wasted-thinking rate (~4% vs 100%). On the short-reasoning benches (AIME / GSM8K / IFEval) it additionally reduces length vs v5-coder. The targeted map never traded length for accuracy — it traded 128e's erratic, sometimes-catastrophic tail for a higher but bounded and productive average.

Methodology / data integrity. Per-problem lengths come from omk_eval token_stats over each bench's per-problem samples archive — lm-eval samples_*.jsonl for the 9-bench, and the native runners' lcb_result.samples.jsonl / mpe_result.samples.jsonl (mirrored into the durable sqlite resume cache) for LCB/MPE. MultiPL-E is tabulated, but its rows measure extracted-code length, not reasoning: its samples store only the final code block (no <think> trace, finish_reasons all terminal, thinking_tokens_est=0), so MPE is a code-conciseness reference — not a rumination metric — on which v6-coder is the most compact. The reasoning-bearing benches (LCB / GPQA / AIME / MATH) carry the rumination signal; MPE tokens there are tokenizer-estimated (the code-extraction path returns no server token count). thinking_tokens_est is 0 across all benches under llama.cpp --reasoning-format deepseek (reasoning is emitted inline, not as a separable block), so the completion-token columns are the length signal. The LCB/MPE token_stats are computed over per-problem rows keyed on doc_id (the native runners now emit it, fixed 2026-05-25) — earlier card revisions briefly collapsed these to n=1; all numbers here are at full n (LCB-55 n=55, LCB-100 n=100, MultiPL-E n=300).

Reproduce

Lineage & provenance

• Base: google/gemma-4-26B-A4B-it (128e, fresh prune — not derived from v4/v5 weights).
• Cohort: v5-coder (C6 _fixed) → C6v3 (C6 v3-data) → C6v3lcb (this, LCB-targeted).
• Tooling: omnimergekit — expert_drop.py, generate_drop_map_multiclass.py, router_shared_upweight.py; eval via omk_eval.py + EVAL_PROTOCOL.
• Eval sampler is frozen GREEDY across the whole cohort for apples-to-apples comparison.