Gemma 4 26B-A4B — Visual Novel (GGUF)

Training

• Method: QLoRA on the MoE experts (Path A "un-fuse"): each fused 3D Gemma4TextExperts weight is split into per-expert nn.Linear leaves so LoRA can target experts.(gate_up|down).N (7885 LoRA modules, 1.88% trainable), then merged + re-fused bit-exact to the canonical Gemma4 layout for GGUF.
• Data: 2,752 VN-protocol turns (traced real play, validity-filtered).
• Schedule: 2 epochs, single B200, 688 optimizer steps, final train loss 0.225.
• Why the experts: training the FFN experts (where phrasing lives) is what moved prose quality — attention-only and dense-FFN-only LoRA gave only a shallow restyle. The experts tune keeps the base model's already-low canned-phrase rate while raising vocabulary diversity and sharpening character voice (see Evaluation).

Evaluation

• The base model is already strong on the protocol, so the win isn't "teaching the format" — it's prose quality at no validity cost. The tune holds 100% validity across 48 held-out scenes (6 seeds) at temperature 1.1, while raising vocabulary diversity (TTR 0.49→0.53) and cutting verbatim cross-scene phrase reuse ~5× (0.019→0.004). Slop-tell density is a wash (2.20 vs 2.11) — the base was never slop-heavy on this list; the tune's gain is voice and variety, not de-cliché-ing.
• Sampling provenance: tune figures are mean over 6 seeds at temp 1.1 (validity over all 48 scenes); the base was sampled once (seed 42) at temp 0.8, its eval default, on the same 8-scene prompt set.

What the metrics mean

• Protocol validity — fraction of generated turns that pass the engine's own validator (vn_validate): well-formed [TOOL: …] calls, parseable choices JSON, and cast-lock (only the locked cast may speak/act). A hard well-formedness gate ("does the turn drive the UI without erroring"), not a taste score.
• Slop-tell density — count of curated "LLM-slop" phrases (stock clichés like "the air hung heavy with unspoken words", "a mix of X and Y") per 1,000 words, via tools/repetition_metrics.py against a hand-curated tell list. Lower = less formulaic. (The list was curated from observed tuned-model failure modes, so it may undercount base-specific clichés — read the base number as a floor, not a like-for-like.)
• Type-token ratio (TTR) — unique words ÷ total words: a vocabulary-diversity proxy. Higher = richer, less word-level repetition.
• Cross-scene trigram reuse — fraction of distinct 3-word sequences that recur across different scenes: a verbatim self-plagiarism proxy. Lower = the model reuses fewer canned spans from scene to scene.

Qualitative read (48 scenes, temp 1.1, seeds 1–6)

• Strengths: genuinely good comedy — per-seed-varied gags with setup / escalation / button (i.e. composing, not memorizing, despite the low 0.225 loss); environmental staging (shows before it tells); distinct, light character voices.
• Weaknesses: romance is the weak suit (stock, on-the-nose, little subtext); the "air heavy / charged with unspoken X" reflex survives the tune but clusters almost entirely in romance scenes; choices lean on an open-up / deflect / stay-silent triad; the occasional garbled line or first↔second-person POV slip.
• Verdict: read it for comedy and cast chemistry; skim the kissing scenes.

Quants

Serving