Views
No views yet
openai/whisper-small on real-world Hindi conversational audio
collected across 102 speakers from India, as part of an AI Researcher Intern
assignment at Josh Talks.| Metric | Value |
|---|---|
| Baseline WER (Whisper-small) | 1.2537 |
| Fine-tuned WER | 0.4028 |
| WER Improvement | ↓ 67.8% |
| Post-processing WER gain | ↓ additional 27.7% |
| Property | Value |
|---|---|
| Total audio | 11.44 hours |
| Speakers | 102 unique speakers across India |
| Segments (after cleaning) | 4,442 |
| Raw segments | 5,941 |
| Train / Val split | 4,093 / 349 |
| Hyperparameter | Value |
|---|---|
| Base model | openai/whisper-small (241.7M params) |
| Learning rate | 1e-5 |
| Effective batch size | 32 (batch 4 × grad accum 8) |
| Epochs | 3 |
| Precision | FP16 |
| Hardware | Kaggle T4 GPU (14.6 GB) |
| Epoch | Train Loss | Val Loss | WER |
|---|---|---|---|
| 1 | 13.22 | 0.657 | 0.546 |
| 2 | 6.98 | 0.471 | 0.435 |
| 3 | 5.07 | 0.414 | 0.403 |
आ आ आ... (100x) → आवगैरा → वगैरह, इदर → इधर| Error Type | Count | % |
|---|---|---|
| Phonetic Confusion | 10 | 40% |
| Spelling Variation | 7 | 28% |
| English Loanword Error | 4 | 16% |
| Filler Word Confusion | 3 | 12% |
| Hallucination / Repetition | 1 | 4% |