Training champion from the whisper-small exploration (Phase-1 holdout score 0.2670 = 0.5·WER + 0.5·CER). Continued / decode-focused variants (e.g. freeze-encoder + higher LR) were explored on top of this recipe; see the sibling base card mmwanje/waxal-whisper-small.

Training notes

Joint multilingual FT (no language tokens in labels — better for Phase 2)
SpecAugment applied on padded mel batches at train time
2×P100 DDP; typical recipe shape: batch=2, grad_accum=4, ~800 steps (see repo experiment log)
Base init: openai/whisper-small (or continue from mmwanje/waxal-whisper-small)

Usage

python

from transformers import WhisperProcessor, WhisperForConditionalGeneration

repo = "mmwanje/waxal-whisper-small-spec_aug_v1"
processor = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(repo)

Code	Language	Whisper lang token
lin	Lingala	lingala
sna	Shona	shona
lug	Luganda	swahili (closest)

Phase 2

Unlabeled clips: run MMS LID (facebook/mms-lid-4017) → force Whisper into {lingala, shona, swahili}. Decode sweeps (no_cond, lang prompts, beams) are inference-only — no retrain required.