KASA-42 — one speech model for 42 Ghanaian languages
Author: Prince Nasamu Alhassan
Overview
A single CTC checkpoint that transcribes
42 Ghanaian language subsets and
identifies which of them is being spoken, without being told. Fine-tuned from
DONDO (
KhayaAI/w2v-bert-…, Apache-2.0),
which covers 11 Northern Ghanaian languages, and extended to the full
ghana-speech corpus with a jointly trained language-ID head.
The 42 configs map to 41 ISO 639-3 codes: Akuapem Twi and Asante Twi share
twi but are separate recording projects and separate rows below.
Results
Book-disjoint test set — no book appears in both training and test.
8,400 utterances, 200 per language.
| Metric | Micro | Macro |
|---|
| WER | 30.2% | 32.5% |
| CER | 10.5% | 11.8% |
| Language ID accuracy | 96.8% mean | 91.5% min, 100% max |
Language ID is the capability DONDO does not have: it conditions on a one-hot
language prefix, so the caller must already know the language. Here it is
inferred, at 96.8% mean accuracy over 42 classes (chance is 2.4%).
Per-language WER ranges from 13.0% (Gonja) to 69.8% (Kabiye). Read those to
the nearest few points, not to one decimal — see the variance note below.
Full per-language results (n=200 each, computed from
records_honest.jsonl with the released scorer)
| Language | WER | CER | Language ID |
|---|
| Gonja (gjn) | 13.0% | 6.0% | 96.5% |
| Birifor Southern (biv) | 15.2% | 4.0% | 100.0% |
| Sisaala Tumulung (sil) | 15.5% | 4.4% | 99.0% |
| Bimoba (bim) | 15.6% | 5.0% | 99.5% |
| Paasaal (sig) | 17.0% | 3.6% | 99.5% |
| Mampruli (maw) | 17.5% | 5.5% | 100.0% |
| Dagaare (dga) | 21.9% | 9.2% | 95.0% |
| Selee (snw) | 22.2% | 3.8% | 99.5% |
| Kasem (xsm) | 22.8% | 11.1% | 97.0% |
| Nawuri (naw) | 22.8% | 10.1% | 98.0% |
| Tampulma (tpm) | 23.4% | 5.6% | 98.5% |
| Lelemi (lef) | 24.1% | 8.8% | 95.5% |
| Dagbani (dag) | 24.5% | 9.1% | 97.0% |
| Konni (kma) | 24.7% | 5.9% | 98.5% |
| Fante (fat) | 25.1% | 5.9% | 93.0% |
| Buli (bwu) | 26.2% | 12.0% | 95.5% |
| Gikyode (acd) | 26.2% | 5.6% | 99.0% |
| Dangme (ada) | 27.7% | 7.8% | 99.5% |
| Sekpele (lip) | 28.0% | 5.7% | 99.5% |
| Anyin (any) | 28.9% | 8.2% | 99.5% |
| Vagla (vag) | 29.2% | 10.8% | 95.0% |
| Deg (mzw) | 29.4% | 11.7% | 95.5% |
| Asante Twi (twi) | 29.5% | 8.6% | 93.5% |
| Kusaal (kus) | 30.2% | 16.3% | 95.5% |
| Fulfulde Maasina (ffm) | 31.2% | 9.7% | 98.5% |
| Chumburung (ncu) | 31.8% | 10.9% | 100.0% |
| Konkomba (xon) | 31.9% | 16.6% | 97.5% |
| Ewe (ewe) | 33.1% | 10.5% | 96.5% |
| Tuwuli (bov) | 37.6% | 14.9% | 93.0% |
| Avatime (avn) | 40.0% | 14.2% | 93.0% |
| Bissa (bib) | 40.9% | 9.5% | 99.0% |
| Tem (kdh) | 41.5% | 8.7% | 98.0% |
| Akuapem Twi (twi) | 41.6% | 16.5% | 92.0% |
| Bassar Ntcham (bud) | 42.6% | 18.6% | 97.5% |
| Sehwi (sfw) | 45.8% | 22.2% | 93.0% |
| Ntrubo (ntr) | 46.9% | 13.3% | 97.0% |
| Hausa (hau) | 47.5% | 31.7% | 98.0% |
| Siwu (akp) | 47.7% | 14.3% | 94.5% |
| Nzema (nzi) | 55.4% | 18.9% | 94.0% |
| Nkonya (nko) | 58.0% | 14.2% | 91.5% |
| Ninkare (gur) | 62.5% | 42.1% | 96.0% |
| Kabiye (kbp) | 69.8% | 25.9% | 95.5% |
Against other systems
Every system scored on the same book-disjoint test set, with the same text
normalisation applied to references and hypotheses alike. Crucially, each row
recomputes both sides over only the languages that system covers — comparing
an average over 34 covered languages against an average over all 42 would
flatter whichever system declined the hardest ones.
| System | Languages | Their WER | Ours | Their CER | Ours |
|---|
| Whisper large-v3 | 42 | 102.3% | 30.2% | 57.3% | 10.5% |
| MMS-1B-all | 34 | 25.7% | 29.2% | 12.1% | 9.7% |
| DONDO (unconditioned) | 8 | 75.9% | 27.6% | 40.8% | 14.1% |
MMS-1B-all beats us on WER by 3.5pp. We beat it on CER by 2.4pp, and cover
eight languages it has no adapter for (both Twis, Dagbani, Dangme, Fante,
Fulfulde and two others). It is also roughly 1.7× our parameter count and does
not do language identification.
That WER/CER split is the informative part. MMS's ratio is 2.1×, ours 3.0× — we
get more characters right and more words wrong, which is a word-segmentation
weakness rather than an acoustic one. The same signature appears in our
per-language table (Bissa 40.9% WER against 9.5% CER). Space prediction is the
obvious place to look next, and we have not looked yet.
Whisper exceeds 100% WER because WER counts insertions: it transcribes these
languages into something else entirely and emits more words than the reference
contains. It was never trained on them.
The DONDO row is not a fair comparison and should not be cited as one.
DONDO conditions on a one-hot language prefix; asr/baselines.py supplies none
and runs it as a plain CTC model, so 75.9% is a lower bound on its ability, not a
measurement of it. Its published figure is ~10.3% average WER on 11 languages.
We report the number for transparency about what we ran, not as evidence about
DONDO. Fixing this needs the conditioning implemented, which the window did not
allow.
Cross-corpus: does it know Kusaal, or one ministry's recordings?
Every number above comes from
ghana-speech. A model trained on one recording
project and tested on the same one cannot distinguish having learned a language
from having learned a studio. So we scored it on an independent Kusaal corpus —
Kusaal ASR Dataset,
81.7 h from Faith Comes By Hearing and Global Recordings Network, forced-aligned
into 9.5 s clips against GILLBT text.
Restricted to the 11 books KASA-42 held out of training, because scripture
corpora share a translation: on any other book the model would already have
trained on the sentences, and only the audio would be new.
| ghana-speech (version 3752) | Kaggle corpus (FCBH / GRN) |
|---|
| Clips | 200 | 2,895 (8.35 h) |
| WER | 30.2% | 37.9% |
| CER | 16.3% | 21.9% |
| Language ID | 95.5% | 99.8% |
Transcription degrades about 25% relative on unfamiliar recordings of the
same language and domain. That is the honest cost of changing recording source,
and it is the number to quote when asking what this model would do on audio from
anywhere else.
Language identification improves, to 99.8%. The LID head evidently keys on
the language rather than the channel — it is more robust to a change of studio
than the transcription is. On 2,895 clips that is a tighter estimate than any
per-language figure elsewhere in this card.
Three caveats we state rather than leave implied:
- Both corpora are scripture. This measures robustness to recording
conditions, not to conversational speech, which remains untested.
- The two may use different Kusaal translations — GILLBT here against
whatever version 3752 uses — so some of the WER gap is vocabulary and phrasing
rather than acoustics.
- The Kaggle clips average 10.4 s against
ghana-speech's ~7.9 s, and longer
clips usually score lower WER because errors amortise over more words. If
anything that understates the degradation.
The leakage experiment, and its negative result
ghana-speech is Bible audio: source_file parses as BOOK.CHAPTER.VERSION.
A random segment split therefore puts adjacent verses — same narrator, same
session — on both sides of the boundary, and in six languages the same book
appears under multiple recording versions, so a random split can place a
byte-identical reference transcript in train and test.
We split by book for that reason, and then tested whether it mattered.
Identical weights were scored on two sets: segments from held-out books, and
segments from seen books that the model was not trained on. Only the second
carries book-level overlap.
| Construction | honest − leaked | 95% CI |
|---|
| Unmatched | −1.1pp | [−1.9, −0.3] |
| Book-diversity matched | −0.7pp | [−1.4, +0.1] |
The interval spans zero. On these weights, book-level overlap gives no
measurable advantage. We report this as a negative result rather than omitting
it.
Two caveats we state rather than leave to the reader:
- This says the model is not memorising books at this scale — one pass over
700 h at ~30% WER — not that leakage cannot occur. A longer-trained or
higher-capacity model may behave differently.
- We did not test chapter-adjacent leakage, which is the strongest form: a
naive split puts consecutive verses from one recording session on both sides.
Our leaked set samples across whole books, which dilutes that.
Book-disjoint splitting remains the conservative default. Our result says it
cost us nothing.
The first construction was confounded — the honest set spanned 4.2 books per
language against the leaked set's 27.2, with 4.9% shorter references — so the
comparison varied content diversity alongside book-seen-ness. The matched row
above removes that.
Sampling variance
Three independent draws of the same test design gave micro WER of 30.3%, 30.0%
and 30.2%, while individual languages moved by up to 7.3pp (Ninkare
57.2% → 64.5%). At n=200 per language, the micro-average is stable to a few
tenths and per-language figures are not. Quote them accordingly.
Training data
ghananlpcommunity/ghana-speech,
CC BY-NC 4.0. 1,411,467 segments, 2,334.9 h across 42 configs.
Training used a temperature-sampled mixture (α=0.5, 40 h per-language cap) over
train-split books only: 700 h from 393,160 segments, which reduces the
largest language's share from 8.5% to 4.9%.
Three things we found in the corpus
id is not unique in 13 of 42 configs, by factors up to 1.52×. Anything
selecting segments by id — the obvious design — silently over-samples those
languages, and the hours accounting reconciles either way, so nothing looks
wrong.- One version code does not imply one narrator. Kusaal is 100% version
3752, yet ECAPA-TDNN speaker embeddings split its books cleanly into Old
Testament (28 books, 39.7 h) and New Testament readers.
- 21 Hebrew characters appear in Ewe and Hausa — Psalm 119's acrostic
stanza headings, 67 occurrences total. They look like an encoding fault and
are not.
Intended use and limitations
Non-commercial use only (CC BY-NC 4.0, inherited from the training data).
This is trained entirely on read scripture: formal, archaic, proper-noun
dense, studio-recorded, one or two narrators per language. It will transcribe
spontaneous conversational speech considerably worse than these numbers
suggest. Every model trained on this corpus has that limitation; ours is not
exempt.
How to use
1import torch, soundfile as sf
2from transformers import SeamlessM4TFeatureExtractor
3from kasa42.asr.model import Kasa42ForCTC, model_state
4from kasa42.asr.dataset import CharTokenizer
5
6tok = CharTokenizer.from_json("vocab.json")
7model = Kasa42ForCTC("KhayaAI/w2v-bert-gjn_maw_gur_dag_dga_kus_lxn_wlx_xon_xsm_en",
8 vocab_size=len(tok), n_languages=42, blank_id=tok.blank)
9model.load_state_dict(model_state("final.pt"))
10model.eval()
11
12fe = SeamlessM4TFeatureExtractor.from_pretrained(model.encoder.name_or_path)
13wav, sr = sf.read("clip.wav") # 16 kHz mono
14feats = fe([wav], sampling_rate=16000, return_tensors="pt",
15 padding=True, return_attention_mask=True)
16with torch.no_grad():
17 out = model(**feats)
18print(tok.decode(out["logits"].argmax(-1)[0][: out["input_lengths"][0]]))
19print("language:", languages[out["lid_logits"].argmax(-1).item()])
An ONNX export is included and verified against the torch model at 150/300/500/900
frames, so it is safe for variable-length audio and runs on CPU.
Training details
| |
|---|
| Base | DONDO KhayaAI/w2v-bert-gjn_maw_gur_dag_dga_kus_lxn_wlx_xon_xsm_en (Apache-2.0) |
| Architecture | w2v-BERT 2.0 encoder + CTC head + mean-pool linear LID head |
| Vocabulary | 96 chars — DONDO's 49 keep their indices and trained head rows, 47 added |
| Blank | [PAD] = 33, not 0 (DONDO's position) |
| Steps | 24,000, batch budget 160 s of padded audio, gradient checkpointing |
| Optimiser | AdamW, lr 5e-5, 500 warmup, cosine decay, encoder frozen 300 steps |
| Precision | bf16 (Hopper) |
| Hardware | 1 × NVIDIA H200 (shared), ~6 h |
Claims we do not make
facebook/mms-tts-kus exists, MMS-1B-all covers Kusaal ASR, and DONDO covers
Kusaal at 13.3% WER. This is not the first Kusaal speech model, and not the
first multilingual Ghanaian ASR. The claims are narrower: broader language
coverage in one checkpoint, language ID that DONDO does not have, and results on
a split that does not leak, with the leakage question actually tested.
See the comparison below for how this sits against MMS, Whisper and DONDO. It is
not first on every metric, and the section says so.
Acknowledgements
Compute provided by
AI Skills and Compute Africa (AISCA), trained on the
Ghana NLP H200. Corpus by the
GhanaNLP community. Base checkpoint by
Paul Azunre (
DONDO).
Please keep derivatives non-commercial and share improvements back with
ghananlpcommunity.
