CDLI SLAM-ASR Luganda Atypical Speech Projector-Only Checkpoint (Epoch 2 Step 107)
Projector-only atypical-speech adaptation checkpoint for SLAM-ASR on the CDLI Luganda atypical speech dataset. The encoder and Sunflower-14B decoder remain frozen; only the linear projector is updated from the ASR-adapted starting checkpoint.
What this repository contains
This Hub repository stores a partial SLAM-ASR checkpoint for use with the
SLAM-LLM codebase. It is not a standalone transformers checkpoint.
- Checkpoint type:
projector_only
- Architecture: Whisper encoder (Sunbird/asr-whisper-large-v3-salt) + linear projector + Sunflower-14B decoder; encoder frozen; LLM frozen; no PEFT adapters.
- Base encoder:
Sunbird/asr-whisper-large-v3-salt
- Base LLM:
Sunbird/Sunflower-14B
- Exported files:
model.pt
Training / evaluation context
- Dataset:
cdli/ugandan_luganda_nonstandard_speech_v1.0
- Evaluation split:
test
- Training speakers: 36
- Validation speakers: 5
- Speaker overlap: No speaker overlap between train and validation/test
Reported metrics
- Normalized WER (JiWER scorer): 59.32%
- Normalized CER (JiWER scorer): 23.07%
- Atypical overall normalized WER: 59.81%
- Atypical overall normalized CER: 23.14%
- Atypical averaged utterance WER: 54.39%
- Atypical averaged utterance CER: 19.19%
Decode settings used for the reported metrics
Test decode used MAX_NEW_TOKENS=200, NUM_BEAMS=4, REPETITION_PENALTY=2.0, NO_REPEAT_NGRAM_SIZE=2, USE_ENCODER_PEFT=false.
Additional results notes
Notebook-style subgroup breakdown on the test split: Mild 48.88% WER, Moderate 52.14%, Severe 63.25%. By disorder: Dysarthria 49.97%, Stuttering 53.65%, Articulation Disorders 54.75%, Voice disorder 68.33%. Average hyp/ref word ratio on test was 92.16%.
Loading notes
Load through SLAM-LLM; this repository stores a partial SLAM-ASR checkpoint, not a standalone Transformers model.
Typical decode flow in this project uses:
examples/asr_luganda/scripts/decode_luganda_sunflower.shUSE_ENCODER_PEFT=true for encoder-LoRA checkpoints- matching LoRA target modules at decode time
Caveats
- This repository stores SLAM-ASR training artifacts intended for research use.
- The checkpoint must be used with the matching SLAM-LLM model code and base components.
- Results can be sensitive to decode settings and evaluation protocol.
Cleaned-Reference Side Analysis
This addendum keeps the archived test predictions fixed and swaps in
the cleaned Luganda references from experiments/luganda/test_cleaned.tsv
where a match exists. It is a reference-quality sensitivity analysis,
not a new decode run.
- Re-score date:
2026-08-21
- Cleaned reference coverage:
1027/1028 test items matched
- Missing cleaned row retained from original reference:
UG022_LUG048
- Matching policy:
keep-original for the single unmatched item
Original archived test metrics
- Atypical averaged utterance WER:
54.39%
- Atypical averaged utterance CER:
19.19%
Cleaned-reference side metrics
- Atypical averaged utterance WER:
47.56%
- Atypical averaged utterance CER:
16.31%
- Delta vs archived WER:
-6.83 points
- Delta vs archived CER:
-2.88 points
- Atypical overall normalized WER:
52.97%
- Atypical overall normalized CER:
20.54%
- JiWER normalized WER:
51.51%
- JiWER normalized CER:
20.30%
Interpretation
The metric change reflects cleaner references, not a model update.
Use the archived metrics for strict apples-to-apples comparison with
the original SLAM-ASR benchmark, and use this section to quantify the
impact of transcript cleanup on the same predictions.
