Whisper-Small Fine-Tuned for Santali Speech-to-Sanlish Transliteration
This repository contains a fine-tuned Whisper-small model for Santali speech-to-Sanlish transliteration.
The model was fine-tuned on a small Santali speech dataset to convert spoken Santali into Roman-script Santali (Sanlish). The model is intended for research on low-resource speech recognition and Santali language technology.
Model Details
- Base model:
openai/whisper-small
- Task: Santali Speech → Sanlish Text
- Model type: Transformer-based encoder-decoder ASR
- Language: Santali
- Output: Sanlish (Romanized Santali)
- Learning rate:
2e-5
- Maximum planned epochs: 25
- Completed epochs: 14
- Training samples: 1,489
- Validation samples: 192
- Test samples: 194
Dataset
The model was fine-tuned using a Santali speech dataset containing paired audio and Sanlish transcriptions.
The dataset was divided into:
| Split | Samples |
|---|
| Training | 1,489 |
| Validation | 192 |
| Test | 194 |
| Total | 1,875 |
The dataset is designed for developing speech technology for Santali, a low-resource language.
Training
The model was fine-tuned from the pretrained Whisper-small model.
Training Configuration
| Parameter | Value |
|---|
| Base model | openai/whisper-small |
| Learning rate | 2e-5 |
| Maximum epochs | 25 |
| Completed epochs | 14 |
| Training samples | 1,489 |
| Validation samples | 192 |
| Test samples | 194 |
Training started from the pretrained Whisper-small model rather than resuming from an existing checkpoint.
Training Progress
The following validation results were obtained during training:
| Epoch | Training Loss | Validation Loss | WER (%) | CER (%) |
|---|
| 1 | 9.889861 | 1.772511 | 90.0101 | 25.8618 |
| 2 | 4.152714 | 1.106039 | 59.6367 | 15.5031 |
| 3 | 2.679677 | 0.931964 | 51.6650 | 12.3360 |
| 4 | 2.070356 | 0.789021 | 45.3078 | 10.6212 |
| 5 | 1.714752 | 0.781359 | 45.3078 | 10.5512 |
| 6 | 1.358998 | 0.809657 | 40.3633 | 9.2913 |
| 7 | 1.101389 | 0.780944 | 40.6660 | 9.3788 |
| 8 | 0.942646 | 0.775277 | 41.0696 | 9.5538 |
| 9 | 0.786113 | 0.758969 | 38.5469 | 8.5214 |
| 10 | 0.641119 | 0.742231 | 39.4551 | 8.9764 |
| 11 | 0.573994 | 0.787470 | 39.7578 | 9.0289 |
| 12 | 0.525331 | 0.778746 | 38.8496 | 8.6614 |
| 13 | 0.422734 | 0.799582 | 39.5560 | 8.7314 |
| 14 | 0.377055 | 0.731823 | 39.6569 | 8.8014 |
The lowest validation WER was observed at epoch 9 (38.5469%).
The lowest validation CER was also observed at epoch 9 (8.5214%).
Test Results
Evaluation on the held-out test set produced:
| Metric | Score |
|---|
| WER | 37.50% |
| CER | 8.2731% |
The test results were saved separately as test_scores.json.
1{
2 "hub_model_id": "thunderboltc/whisper_sanlish_mnx_lr2e5",
3 "learning_rate": 2e-05,
4 "num_train_epochs": 25,
5 "test_wer": 37.5,
6 "test_cer": 8.273092369477911,
7 "n_train": 1489,
8 "n_val": 192,
9 "n_test": 194
10}
