A streaming automatic speech recognition model for Luganda, Shona, and Swahili (with English retained), fine-tuned from nvidia/nemotron-3.5-asr-streaming-0.6b — a FastConformer Cache-Aware RNN-Transducer (~600M parameters). By CraneAI Labs.

The model transcribes conversational and read speech in real time (cache-aware streaming) and is conditioned on a language-ID prompt.

What's new in 1.1. Version 1.0 was trained on a single 16 GB T4, which forced a partial fine-tune (only the top 12 encoder layers). 1.1 was trained on an NVIDIA L40 (48 GB), which let us fine-tune the full encoder (all 24 layers plus the prediction and joint networks, ~633M trainable parameters) in bf16 and re-tune the learning rate for the larger trainable surface. The result is a sizable accuracy gain across all three African languages — Luganda and Shona improve the most (Luganda WER 33.8% → 20.9%, Shona WER 37.7% → 27.3%).

Results

Evaluated on a frozen, held-out set of 50 clips per language (excluded from training by both utterance ID and speaker ID). WER = word error rate, CER = character error rate (both computed under a shared normalization: NFKC, lowercased, punctuation-stripped, whitespace-collapsed), and Gemini = a meaning-preservation score from 1–5 (LLM-as-judge), which better reflects intelligibility for morphologically rich Bantu languages where word-level WER is harsh.

Language	CER ↓	WER ↓	Gemini (1–5) ↑
Luganda (`lg`)	5.1%	20.9%	4.95
Shona (`sn`)	6.0%	27.3%	4.50
Swahili (`sw`)	7.3%	25.1%	4.35
English (`en`)	3.6%	9.1%	4.65

Improvement over 1.0. All three African languages improve on both CER and WER:

Language	CER (1.0 → 1.1)	WER (1.0 → 1.1)
Luganda	8.2% → 5.1%	33.8% → 20.9%
Shona	8.1% → 6.0%	37.7% → 27.3%
Swahili	8.1% → 7.3%	27.5% → 25.1%
English	3.2% → 3.6%	7.4% → 9.1%

English sees a small retention cost (the price of unfreezing the full encoder at a higher learning rate); its meaning-preservation score stays high (4.65).

Starting point (before fine-tuning). On these African languages the base model is effectively unusable — roughly 100% WER: Luganda 99.7%, Shona 101.0%, Swahili 102.5% (CER 59–76%). Fine-tuning brings them to the figures above. (English is already supported by the base model and stays strong.)

How WER/CER are calculated. Predictions and references are passed through one shared text normalizer — Unicode NFKC, lowercasing, punctuation removal, and whitespace collapsing — and then scored with jiwer, aggregated across the whole eval set (total edits ÷ total reference units), not averaged per clip.

Why normalize. Without it, WER counts casing and punctuation differences as full word errors even when the spoken content is transcribed correctly — and the references for these languages carry sentence-case and punctuation conventions the model isn't asked to reproduce. Normalization removes that bookkeeping penalty so the number reflects recognition quality rather than formatting.

Note on metrics: CER and the Gemini meaning-score are the most faithful quality signals for these languages. WER runs higher than CER largely because rich agglutinative morphology and word-boundary conventions penalize the word-level metric even when the transcription is intelligible.

Training

Method: full fine-tune — the entire encoder (all 24 layers) is trained together with the prediction network and joint network (~633M trainable parameters, 99% of the model).
Optimizer: 8-bit AdamW, learning rate 3.46e-4 (selected by a coarse-to-fine learning-rate sweep on the frozen eval set), cosine schedule with 200 warmup steps, 6,000 steps.
Precision: bf16-mixed. Hardware: a single NVIDIA L40 (48 GB).
Decoding/inference: cache-aware streaming, fp32, attention context [56, 13] (~1.12 s lookahead).

Training data

Language	Approx. hours
Luganda	~13.7h
Shona	~15.8h
Swahili	~9.2h
English	~2.0h (retention)

Data is read and conversational speech drawn from openly available African-language speech corpora; a small English slice is included to retain English performance.

Intended use

Transcription of Luganda, Shona, Swahili, and English speech, including low-latency streaming applications.
Suitable as a strong, deployable baseline for these languages and as a starting point for further fine-tuning.

Limitations

Results are reported on 50 clips per language — directional, not a large-scale benchmark.
Cache-aware streaming inference is fp32-only.
Best results use the language-ID prompt set to auto-detect; Luganda and Shona do not have a dedicated prompt slot, so auto-detect is recommended for them.
The full encoder is fine-tuned on a few tens of hours per language, so very out-of-domain acoustics (heavy background noise, far-field/distant microphones, dense code-switching) are still only partially handled.
Not validated for safety-critical or medical/legal transcription.

Usage

python

import nemo.collections.asr as nemo_asr

model = nemo_asr.models.ASRModel.restore_from("crane-nemo-asr-1.1.nemo")
transcripts = model.transcribe(["audio.wav"])
print(transcripts)

For cache-aware streaming inference, use NeMo's examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py with compute_dtype=float32, att_context_size=[56,13], and target_lang=auto.

License

This model inherits the NVIDIA Open Model License from its base model.

CraneAILabs/crane-nemo-asr-1.1

Crane Nemo ASR 1.1