GemmaWiki: Gemma 4 26B-A4B LoRA Adapters for Wiki Editing
This is an independent, unofficial project. It is not affiliated with, endorsed by, or
produced by Google or Google DeepMind. "Gemma" is a trademark of Google DeepMind, used
here only to identify the base model this project fine-tunes.
Status: v1 is released and usable. It performs well in evaluation (see Results), with one main rough edge: on very well-known book series, generated pages can drift toward the base model's own pretraining knowledge instead of sticking to the supplied chapter (see Known limitations). Treat it as a solid pilot rather than a finished product. A broader corpus and stronger chapter-grounding are planned next.
LoRA adapters (r=32, attention-only: q_proj/k_proj/v_proj/o_proj) fine-tuning Gemma 4
26B-A4B, a mixture-of-experts model, as an automated fan-wiki editor. Given a book chapter
and, when one exists, an entity's current wiki page, the model produces one of three
outputs: a new page, an updated page, or a classification that no change is needed, using
only information available up to that chapter. Two independently trained variants are
provided here: one starting from the base pretrained model, one starting from Google's
instruction-tuned checkpoint. Both ship as LoRA adapters and as ready-to-run GGUF
quantizations.
Structure
Every checkpoint is a standard PEFT adapter directory (adapter_config.json plus
adapter_model.safetensors).
Base-backbone adapters, adapters- base/, from google/gemma-4-26b-a4b
run1-checkpoint-{25,50,75}: initial run, trained on a single franchise from the
corpus only, an unintentional data-ordering artifact. Kept for the record, not the
recommended checkpoint.
run2-checkpoint-{025..300}, run2-checkpoint-304, run2-final: the recommended
series. Warm-started from run1-checkpoint-75, retrained on a corpus-proportional balance
across six popular fantasy and young-adult book series, covering page creation, page
update, and chapter-triage classification. run2-final (identical weights to
run2-checkpoint-304) is the finished model, trained for one epoch over the full balanced
dataset.
Instruction-tuned-backbone adapters, adapters- it/, from google/gemma-4-26b-a4b-it
run3-checkpoint-{025..300}, run3-final: the same training recipe as run2, applied
to Google's instruction-tuned checkpoint instead of the plain pretrained one. run3-final
is the finished model. Because this backbone already understands conversational
formatting, it can be prompted two ways: raw (plain text, matching the training format
directly, the recommended default) and chat (through the model's own chat template). See
Results and How to Use below for the tradeoffs between them.
GGUF quantizations
Ready-to-run GGUFs for both variants (merged into their respective base models in bf16, then
converted with llama.cpp). Root-level files, so they work directly with llama.cpp, LM
Studio, and ollama run hf.co/Calplus/GemmaWiki-Gemma-4-26b-a4b:Q4_K_M.
File
Size
Notes
GemmaWiki-Gemma-4-26b-a4b.Q4_K_M.gguf
16.8 GB
Recommended deployment quant, base-backbone (5.32 bpw, MoE expert tensors fall back to q8_0/q4_0 where k-quant blocks don't fit; benchmarked, see Results)
GemmaWiki-Gemma-4-26b-a4b.Q5_K_M.gguf
19.1 GB
Higher-fidelity middle ground, not separately benchmarked
GemmaWiki-Gemma-4-26b-a4b.Q8_0.gguf
26.9 GB
Near-lossless reference (8.51 bpw), not separately benchmarked
GemmaWiki-Gemma-4-26b-a4b.F16.gguf
50.5 GB
Full-precision master, use it to make your own quants
GemmaWiki-Gemma-4-26b-a4b-it.Q4_K_M.gguf
16.8 GB
Instruction-tuned-backbone equivalent (partially benchmarked, see Results)
GemmaWiki-Gemma-4-26b-a4b-it.Q5_K_M.gguf
19.1 GB
Higher-fidelity middle ground, not separately benchmarked
GemmaWiki-Gemma-4-26b-a4b-it.Q8_0.gguf
26.9 GB
Near-lossless reference, not separately benchmarked
GemmaWiki-Gemma-4-26b-a4b-it.F16.gguf
50.5 GB
Full-precision master, use it to make your own quants
Decoding note: use sampling with a repetition penalty (llama.cpp defaults are fine). Pure
greedy decoding can fall into repetition loops on long generations, a base-model (Gemma 4
26B-A4B) behavior that fine-tuning does not remove.
Results
All figures below come from an automated evaluation suite comparing generated text against
data withheld from training. No LLM judge is involved in any of these numbers: every metric
is an exact-match, classification, or statistical computation. Three request types are
covered: page update, page creation, and chapter-triage classification.
All of this evaluates the model within its training distribution (one franchise was held out entirely); a qualitative caveat about very well-known series is noted under Known limitations.
Component metrics:
D score: converts held-out cross-entropy loss into a 0 to 100 scale
(100 * e^(-loss)), representing how much probability the model assigns to the actual
correct continuation on wiki-editing examples it never trained on.
I stability: 100 minus the rate of degenerate outputs across a self-feeding rollout,
where the model repeatedly edits its own prior output as new chapters arrive, matching how
it would be used in production.
Format: percentage of outputs with correct structure (front matter and headings for
page edits, valid JSON for classification).
Degeneration: percentage of outputs that collapsed into repetition loops or character
floods. Lower is better.
Macro-F1: macro-averaged F1 for the three-way classification task (new page, update,
no change).
Composite combines these into one score: 50% D score, 25% I stability, 15% macro-F1, and
10% split between format and inverted degeneration rate. The weighting favors the properties
that most directly show the model has learned the task and holds up under realistic,
repeated use, with classification accuracy and basic output hygiene as secondary factors.
Model
Composite
D score
I stability
Format
Degeneration
Macro-F1
Gemma 4 26B-A4B (untrained)
63.4
63.1
98.4
19.2
3.1
9.5
Gemma 4 26B-A4B-it (untrained, raw prompt)
31.9
14.6
70.3
69.2
56.2
9.3
Gemma 4 26B-A4B-it (untrained, chat prompt)
58.7
41.3
100.0
69.2
0.0
30.4
run1
67.7
65.7
100.0
100.0
3.1
0.0
run2 (recommended)
75.7
68.0
100.0
100.0
0.0
44.6
run3, raw prompt
68.4
58.6
98.4
73.1
0.0
38.8
run3, chat prompt
69.0
60.8
100.0
34.6
3.1
46.7
run2, Q4_K_M
73.5
67.2
100.0
100.0
0.0
32.4
run3, Q4_K_M, raw prompt
67.7
57.1
100.0
76.9
0.0
35.2
run3, Q4_K_M, chat prompt
73.9¹
not measured
100.0
46.2
0.0
30.9
¹ D score for this configuration was not measured (evaluation compute went to the
configurations above first); composite renormalized over the remaining three weighted
terms.
Both fine-tuned variants show a large, consistent gain over their untrained starting point
on every axis that measures task competence. The untrained instruction-tuned checkpoint in
particular is unstable under repeated use (70.3 I stability, 56.2% degeneration in raw
prompting), and fine-tuning resolves both.
General knowledge (MMLU, 342 held-out multiple-choice questions) is not part of the
composite above since it measures a different property than wiki-editing competence. For
reference: base 73.7%, run1 74.9%, run2 75.1%, run2 Q4_K_M 74.9% (quantization costs 0.2
points), untrained instruction-tuned backbone 46.5%, run3 39.8%. The lower figures for the
instruction-tuned-backbone variants reflect the raw, non-chat-formatted prompting style this
test uses rather than knowledge loss specifically.
Deployment throughput (Q4_K_M, single GPU): run2 60.2 tok/s with full GPU offload, 11.9
tok/s with partial offload; run3 67.7 and 9.5 tok/s respectively.
Known limitations
Page content can drift toward the base model's own knowledge of very famous series. In held-out evaluation the model writes from the chapter it is given, and hand-review found no spoiler leakage. In field testing on one of the most widely known franchises in the corpus, some pages (mostly newly created ones) mixed in details Gemma already knew from its pretraining rather than sticking to the chapter. Updates to an existing page stayed closer to the source. Fine-tuning dampens this familiarity but does not erase it, so review new pages for well-known series before publishing. Stronger chapter-grounding is a goal of the next training round.
Input-shape specialization. Both fine-tuned variants are specialized to two request
shapes: chapter plus prior page, and chapter plus classification question. A request
outside either shape typically produces no output rather than an attempted answer, at any
prompt length. This is a direct consequence of narrow, single-purpose LoRA training.
Calling code should decide which request shape applies before invoking the model; see How
to Use.
General knowledge is out of scope for the instruction-tuned-backbone variant when
prompted without a chat template. See Results for the numbers and context; this does not
indicate an issue with the wiki-editing task itself.
run1 is kept for reference only. Its training data was accidentally skewed toward a
single franchise; use run2 or run3 instead.
run3's Q4_K_M quantization has one unmeasured configuration (chat-prompt D score, see
footnote in Results). The configurations that were measured already answer the question of
whether quantization holds up.
This suite measures task competence, learned confidence, and mechanical output quality.
It does not include a separate qualitative audit of factual grounding or spoiler avoidance
in generated prose. As with any generative model, review output before publishing it to a
live wiki.
How to use
Raw vs. chat mode (run3 only)
run2 has one calling convention: plain text, matching its training format. run3 supports
two: raw, plain text identical to run2's convention and the recommended default, and
chat, through the model's chat template. Raw mode scores modestly higher on learned
confidence and much higher on output-format consistency (see Results). Chat mode is a
reasonable alternative if the calling code already expects and strips conversational
wrapping around the structured output.
Request shapes
The model supports exactly two request types:
Page update or creation: a book chapter, plus the entity's existing wiki page if one
exists. The model returns the new or updated page.
Classification: a book chapter and a question asking whether given entities' pages
need a new page, an update, or no change. The model returns a JSON object mapping entity
names to one of the three labels.
Production integration
Because the model is specialized to exactly these two shapes, treat it as a narrow
component inside a larger pipeline rather than a general-purpose agent:
Decide, in surrounding code, which of the two request shapes applies before calling the
model. Do not ask the model to infer this itself.
Validate output before publishing it: confirm front matter and headings for page edits, or
that the response parses as JSON for classification.
Treat empty output as a signal that the request did not match a recognized shape, not as a
successful no-op.
Loading the adapters (bf16, transformers plus PEFT)
ollama run hf.co/Calplus/GemmaWiki-Gemma-4-26b-a4b:Q4_K_M
Or load any of the files listed above directly with llama.cpp or LM Studio.
Future work
Broaden the training corpus beyond the current six franchises.
Strengthen chapter-grounding so pages rely less on the base model's prior knowledge of famous series.
Revisit the instruction-tuned variant's chat-mode formatting, which lags raw mode in the current results.
Training details
Base architecture: Gemma 4 26B-A4B, a mixture-of-experts transformer. LoRA rank 32,
attention projections only (q_proj, k_proj, v_proj, o_proj). Hardware: bf16 LoRA
training with FSDP across 4 A100 GPUs.
Training data: sourced from 2,287 book chapters across 7 franchises, scraped into 53,963
wiki pages and cleaned down to 32,234 articles relevant to the source books. Six of the
seven franchises were combined into the corpus-proportional training mix described above;
the seventh was held out entirely to test generalization to a series the model never
trained on. The model was trained on a corpus of 245,509 examples, automatically audited
beforehand to catch spoiler leaks in the underlying wiki-page targets.
License
Apache 2.0, inherited for the adapters and the GGUF conversions in this repository. The
base model itself remains subject to Google's own Gemma terms of use.
