Bilingual, code-switch-aware, phone-robust speech recognition for Asante Twi / Akan and English, for Ghanaian customer-service voice agents (8 kHz telephony). CTC fine-tune of facebook/w2v-bert-2.0 (580M, MIT) on real conversational + read Ghanaian speech + English, with phone-codec + SpecAugment augmentation.

v0.5 adds 8 kHz μ-law codec + SpecAugment augmentation over v0.4, cutting phone-codec WER substantially while slightly improving clean — the phone gap is nearly closed.

Results — WER by source (dev*)

Source	Type	v0.4 WER	v0.5 WER	v0.5 +KenLM
BibleTTS unseen	Twi read	9.8	9.1	8.0
BibleTTS — 8 kHz phone	Twi read, telephony	11.7	9.5	—
WAXAL Akan	conversational + code-switch	27.0	26.3	25.8
WAXAL — 8 kHz phone	conversational, telephony	33.0	27.9	—
FLEURS ‡	English	11.7	11.7	6.4 ‡
Ashesi FISD †	Twi read, fintech	1.2	1.0 †	—

* These are dev-set numbers. A speaker-disjoint, held-out test split is in construction and will be published alongside the next release. We report the caveats below rather than omit them, because two of these rows are not generalisation claims.

† Ashesi FISD is in-domain and should not be read as a headline. The corpus authors state that speakers and phrases in the 10% eval split may also appear in the 90% train split; we measured 100% transcript overlap and 100% speaker overlap with our training data. The 1.0 WER is reported for completeness and continuity with v0.4, not as evidence of generalisation.

‡ Treat the KenLM-fused English number as optimistic. Our bilingual LM corpus is counted from the training text, which overlaps ~77% of this FLEURS dev slice, so the 11.7 → 6.4 improvement is substantially the language model recognising sentences it was built from. The unfused English number (11.7) is the honest one; Twi-read 9.1 → 8.0 and conversational 26.3 → 25.8 rest on non-overlapping corpora and stand.

The defensible headline is the phone-band conversational result: BibleTTS 9.1 clean / 9.5 telephony, WAXAL 26.3 clean / 27.9 telephony — measured on named, held-out sets with no train overlap.

Phone robustness is the headline: conversational-phone WER 33.0 → 27.9 (−15% rel), read-phone 11.7 → 9.5; clean held/improved. BibleTTS phone (9.5) ≈ clean (9.1) — nearly codec-invariant, which an 8 kHz call pipeline needs. A bilingual KenLM 3-gram shallow fusion (pyctcdecode) lowers WER further at decode time — English 11.7 → 6.4, Twi-read 9.1 → 8.0, conversational 26.3 → 25.8 (see decode_lm_eval.py).

Training data

Twi read: BibleTTS Asante (ɛ/ɔ-corrected) + Ashesi/Nokwary Financial Inclusion + Common Voice Twi.
Conversational + code-switch: WAXAL Akan (google/WaxalNLP, aka_asr, CC-BY-4.0).
English: FLEURS en_us (CC-BY).
~109 h, char-level CTC, ɛ/ɔ preserved, NFC-normalized, untoned.

Training

w2v-BERT 2.0 CTC, 12 epochs on 8×A100, batch 4, lr 1e-4, MAX_S=25. Augmentation (TUBASTT_AUG_PROB=0.4): per-utterance 8 kHz G.711 μ-law round-trip + SpecAugment on train only. Code: models/asr/tubastt/. Per-source numbers above are the reported results; we do not quote a single combined dev WER, because our training-time eval metric pads accumulated logits in a way that inflates it — the per-source evaluation is the trustworthy measurement.

Usage

python

import torch, soundfile as sf
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor
proc = Wav2Vec2BertProcessor.from_pretrained("neriqlabs/tubastt-twi-en-asr-w2vbert-v0.5")
model = Wav2Vec2BertForCTC.from_pretrained("neriqlabs/tubastt-twi-en-asr-w2vbert-v0.5").eval()
wav, sr = sf.read("utt.wav")  # 16 kHz mono
feats = proc.feature_extractor(wav, sampling_rate=16000, return_tensors="pt").input_features
print(proc.batch_decode(model(feats).logits.argmax(-1))[0])

Limitations & next

Conversational WER (~26% clean / 28% phone) still above read speech — the honest production bar. Next lever: pseudo-label self-training on unlabeled Ghanaian audio (UGSpeech Akan ~1000 h).
WAXAL is Akan (Twi is a dialect); slight dialectal drift vs pure Asante Twi.
English accent skews to FLEURS (generic); Ghanaian-English polish planned.

Base: facebook/w2v-bert-2.0 (MIT). Built by Neriqlabs.