Vietnamese Zipformer ASR
Model description
Model type: Automatic speech recognition (ASR) — streaming-capable transducer (Zipformer encoder + stateless decoder + joiner), exported as ONNX and quantized to INT8 for deployment.
Languages: Vietnamese (primary); handles Vietnamese–English code-switched speech.
Training framework: Icefall (k2 + Lhotse); pruned RNN-T loss; optional CTC; ScaledAdam + Eden scheduler; mixed precision (FP16/BF16).
Evaluation (results-4): WER reported on VietCasual test sets (combined A+B), normalized transcripts, jiwer-based WER.
Model variants and performance
| Variant | Checkpoint | Overall WER (%) | Notes |
|---|
| exp7 | epoch-4-avg-4.int8 | 8.22 | Best WER in results-4 |
| exp7 | epoch-10-avg-10.int8 | 8.43 | |
| exp7 | epoch-15-avg-15.int8 | 8.43 | |
| exp7 | epoch-1-avg-1.int8 | 9.80 | Early epoch |
| zipformer-6000h | epoch-20-avg-10.int8 | 21.93 | ~6k-hour pretrain; different data/domain |
| viet_iter | epoch-12-avg-8.int8 | 25.79 | Different training setup |
- Recommended for VietCasual: exp7 with epoch-4-avg-4.int8 (8.22% WER).
- Test set: VietCasual (vietcasual_test_A_1000 + vietcasual_test_B_1000), casual Vietnamese and code-switched English; transcripts normalized (e.g. soe_vinorm, punctuation cleaning).
Intended use
- Primary: Vietnamese ASR and Vietnamese–English code-switched speech (e.g. podcasts, interviews, casual dialogue).
- Deployment: On-device or server with sherpa-onnx (INT8 ONNX); supports streaming.
- Out-of-scope: Formal written text read-aloud only; other languages without adaptation; high-noise or heavily accented speech without further fine-tuning.
Training and evaluation data
- Training (representative): Vietnamese cuts (e.g. vi_cuts_train), optionally VietCasualSpeech, VietCetera, VietSuccess; LibriSpeech or mixed data for pretraining/fine-tuning.
- Evaluation (this card): VietCasual test A+B; WER computed with jiwer on normalized reference vs hypothesis.
Evaluation setup
- Metric: Word error rate (WER), case-sensitive on normalized text.
- Tooling: sherpa-onnx transducer decoder;
run_wer_casual.py (or equivalent); combined wer_results.txt / wer_errors.txt per checkpoint.
- Outputs: Per-utterance
dataset_tag:wav_name | reference | hypothesis | wer_pct; overall WER reported at top of wer_results.txt.
Limitations and bias
- Performance depends on domain and normalization; code-switched English segments can have higher error rates.
- INT8 quantization may slightly affect accuracy vs FP32.
- Not tested for robustness to strong accents, dialects, or very noisy environments in this card.
How to use (inference)
Example layout for sherpa-onnx (e.g. under exp7/epoch-4-avg-4.int8/):
encoder-<name>.onnxdecoder-<name>.onnxjoiner-<name>.onnxtokens.txt
Use sherpa-onnx Transducer API with the above files and your 16 kHz (or model-native) audio. See project scripts (e.g. utils/WER/run_wer_casual.py) for batch WER.
Citation
If you use these models or results-4 metrics, please cite the training framework (e.g. Icefall/k2, Lhotse) and this card’s WER table and evaluation setup.
Summary
| Field | Value |
|---|
| Best WER (results-4) | 8.22% (exp7, epoch-4-avg-4.int8) |
| Test set | VietCasual A+B |
| Model form | ONNX INT8 transducer (Zipformer) |
| Languages | Vietnamese, Vietnamese–English code-switch |
