Quantised builds of a conversational Welsh model small enough to run offline on a laptop, a phone or a Raspberry Pi class board with llama.cpp or LM Studio. Q6_K needs about 7 GB of memory, Q4_K_M about 5 GB. No internet, no data centre, just Welsh.
GGUF quantisations of
EryriLabs/gemma-4-e4b-cymraeg-v4,
a conversational Welsh fine-tune of google/gemma-4-E4B-it that retains
English.
quant
file
size
sha256
Q8_0
gemma4-e4b-cymraeg-v4-Q8_0.gguf
see MANIFEST.json
see MANIFEST.json
Q6_K
gemma4-e4b-cymraeg-v4-Q6_K.gguf
see MANIFEST.json
see MANIFEST.json
Q4_K_M
gemma4-e4b-cymraeg-v4-Q4_K_M.gguf
see MANIFEST.json
see MANIFEST.json
Q6_K or better is the recommendation. Each quantisation was validated at
temperature 0 against a corruption-prone prompt subset; sha256 sums and the
corruption counts for every quant are in MANIFEST.json. Read that evidence
before trusting the Q4_K_M build for anything where Welsh accuracy matters ,
it is provided for tight memory budgets, not as the default choice.
--jinja is required. Without it llama.cpp substitutes a generic chat
template, the Gemma 4 thought channel is never closed, and replies come back with
empty content after the whole token budget is spent. LM Studio has shown the
same failure; if answers arrive empty, this is the cause.
Keep thinking off (the shipped chat template's default).
Recommended sampling:
setting
value
temperature
0.3
top_p
0.9
min_p
0.05
repeat_penalty
1.05
Fraud-related facts (contact numbers, organisation names, reporting routes)
should be supplied through the system prompt or retrieved context rather than
relied on from the model's memory: see README-stopthinkfraud.md.
Evaluation
Blind LLM-as-judge scoring (Opus), identical prompts and settings for every arm
(nf4, thinking off, greedy, 400 max tokens, no system prompt). The judge is not
a native speaker; structural findings (language choice, loops, checkable facts)
are reliable, fine idiom judgements less so: native review welcome.
Single-turn, 40 Welsh prompts
model
replies in the requested language
fluency /5
grammar /5
task /5
stock gemma-4-E4B-it
47.5 %
2.46
2.38
1.45
v3 (SFT only)
97.5 %
3.87
3.59
2.33
v4 (CPT 43.8M tokens + SFT)
100 %
3.68
3.76
2.70
Pairwise v4 vs v3 (blind): 23 wins, 13 ties, 4 losses. English is fully
retained (fluency 5.0, task 4.95: best in the family). Invented Welsh words
roughly halved versus v3; the corpus-installed mutations (i Gaerdydd,
i Fangor, dy fore di) are v3's exact failures now correct.
Multi-turn (14 fixed turns, history kept, sampling as recommended above)
model
conversation score /5
stock
1.97
v3
3.35
v4
3.99
Quantisation: Q6_K scored highest of the three quants under llama.cpp
(fluency 4.42, grammar 4.36); Q4_K_M showed no measurable damage; all three
quants: 0 corrupt tokens on the corruption-prone subset.
Known limitations: occasional repetition loops on long open-ended answers
(mitigated by repeat_penalty); factual recall (dates, statistics) is weak ,
supply facts via the system prompt / retrieval; when given long retrieved
context it can over-quote it. Fraud-safety facts must not be relied on from
memory. Full judgement report and per-response scores:
eval/judgement_e4b_v4.md, eval/scores_e4b_v4.jsonl in the base repository.
Comparison with other open Welsh models: see the note on the base
repository: techiaith (Bangor University) and BangorAI publish the only other
open Welsh LLMs to compare against, and their open datasets are part of this
model's training data, with thanks. A head-to-head against
BangorAI/Mistral-7B-Cymraeg-Welsh-v2 on this exact protocol is in the section
below.
Head-to-head with BangorAI Mistral-7B-Cymraeg-Welsh-v2
BangorAI/Mistral-7B-Cymraeg-Welsh-v2
is the strongest open Welsh model we know of and the only open comparison in the
7B class, so it is the bar this model is measured against. It is a genuinely good
Welsh model, and everything below should be read as a measurement against a high
bar rather than as a criticism of it. Diolch to BangorAI for publishing it.
It is not a like-for-like comparison, and it is not meant to be:
this model (v4)
BangorAI 7B v2
parameters
6.9B total, ~4B effective per token
7B dense
Welsh continued pretraining
43.8M tokens, 1 pass
MADLAD Welsh, 2 epochs, on the order of 412M token-passes
ratio
1x
~9.4x the Welsh tokens, ~1.75x the active parameters
BangorAI was run with its own documented Welsh system prompt, which is its
intended interface, in the single-turn arm and in the multi-turn sys arm. The
multi-turn nosys arm is the properly matched comparison. Sampling,
quantisation, prompts and token limits were identical throughout.
Single-turn, 40 Welsh prompts plus 20 English prompts
arm
replies in the requested language
Welsh fluency /5
Welsh grammar /5
task /5
collapses
English task /5
v4
40/40
3.68
3.76
2.70
5
4.95
BangorAI 7B v2
37/40
4.85
4.65
3.35
0
4.20
Pairwise on the same 40 prompts: BangorAI 23 wins, 13 ties, v4 4 wins. That is
the mirror image of v4 against v3. BangorAI's Welsh essentially never breaks:
zero degeneration loops and zero corrupt word-forms anywhere. v4's four wins are
the three prompts that ask for English output, which BangorAI answers in Welsh,
plus one conversational prompt. v4 leads on task in translation (3.50 against
2.00) and assistant chat (3.43 against 2.86); BangorAI leads everywhere else.
Multi-turn, 14 fixed turns with history kept
arm
fluency
grammar
relevance
conversational
memory
mean
v4 / sys
4.07
4.14
3.21
3.36
3.69
3.70
v4 / nosys
4.29
4.14
3.79
3.86
3.85
3.99
BangorAI / sys
4.79
4.93
3.36
3.00
2.92
3.80
BangorAI / nosys
4.64
4.93
3.21
2.93
3.00
3.74
Per-turn wins, ties and losses:
condition
v4
tie
BangorAI
sys vs sys (each with its own intended prompt)
5
2
7
nosys vs nosys (matched)
5
4
5
combined, 28 turns
10
6
12
v4 wins the turns that hold a conversation together: the opening greeting, the
summary, the close (both v4 conditions produce an eleven-turn callback,
"Mwynha dy daith i Aberystwyth", which no BangorAI condition does), and above
all turn 11, the request to switch to English. v4 switches cleanly and
translates faithfully in both conditions; BangorAI stays in Welsh in both, and
under its own system prompt it states in Welsh that it can only help in English.
BangorAI wins the turns that reward content and polish: the tips, the facts, the
joke, the fraud advice and the poem.
Verdict in three lines. BangorAI wins overall and wins Welsh polish
decisively: it is more fluent, more grammatical and never degenerates, and if you
want the best open Welsh output and can run a 7B, use it. The conversation is far
closer, and in the matched condition v4 takes it on the arm mean (3.99 against
3.74) and draws on turns, so roughly 4B effective parameters trained on a tenth of
the Welsh fights a 7B to a draw over fourteen turns. What the extra scale and data
buy is surface quality and degeneration resistance; what they do not buy is
memory, register mirroring or switching language on request, which is why a small
bilingual model still has a place on edge devices.
The 12B flagship gemma-4-12b-cymraeg-v6
has since won a pre registered 100 prompt conversational benchmark against BangorAI's
7B, 66 to 33; see that repo for method and results.
Caveats. The judge is an LLM (Opus) and not a native Welsh speaker;
structural findings are reliable, fine idiom judgements are not, and a non-native
judge is least reliable exactly where BangorAI wins most, so the fluency gap is
more likely understated than overstated. Sizes and Welsh token counts are not
matched. BangorAI ran through its own system-prompt interface, so only the
multi-turn nosys comparison is matched on prompt text. The rubric was written
for a friendly conversational model, which favours v4 on register and warmth.
Sample sizes are small. Full report and per-item scores:
eval/judgement_v4_vs_bangorai.md and eval/scores_v4_vs_bangorai.jsonl in the
base repository.
Limitations
Quantisation below Q6_K measurably increases degradation inside Welsh text;
read the corruption counts in MANIFEST.json before choosing Q4_K_M.
Welsh costs more tokens per word than English for this tokeniser, which
reduces the effective context length available for Welsh text relative to
English.
Fraud-related facts (numbers, organisation names, reporting routes) can be
misremembered or go stale; supply them through the system prompt or retrieved
context rather than relying on the model's memory.
No tool-calling and no guarantee of factual accuracy.
Licence and attributions
Apache-2.0, inherited from google/gemma-4-E4B-it (trained from the
unsloth/gemma-4-e4b-it mirror). Commercial use is permitted. Training sources
and their individual licences are listed in DATA-CARD-v4.md and
README-stopthinkfraud.md in the
base repository;
where a source licence requires attribution (CC-BY-4.0, ODC-By 1.0, OGL v3),
that requirement passes through to users of these files.
