This model is a fine-tuned version of openai/whisper-medium on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

Wer: 24.5705
Cer: 9.3880
Real Time Factor: 0.0531

Quick Inference

python

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

E1 (Baseline): Standard training configuration.
E2 (Noise): Training with background noise augmentation.
E3 (Speed): Training with speed perturbation augmentation.
E4 (SpecAug): Training with SpecAugment (time and frequency masking).
E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: OptimizerNames.ADAMW_TORCH_FUSED
lr_scheduler_type: SchedulerType.LINEAR
num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.23	0.6933	0.6139	40.78	16.18	1.49e-04	6.36
500	0.46	0.9767	0.8149	47.59	26.86	2.99e-04	6.31
750	0.68	0.7091	0.7418	41.24	17.77	2.91e-04	5.07
1000	0.91	0.6460	0.7217	37.89	15.42	2.82e-04	5.18
1250	1.14	0.4395	0.6853	36.05	16.92	2.73e-04	4.28
1500	1.37	0.3875	0.7011	37.08	18.51	2.64e-04	2.90
1750	1.59	0.3301	0.6582	35.70	17.54	2.55e-04	3.33
2000	1.82	0.2798	0.6510	33.88	15.94	2.46e-04	2.80
2250	2.05	0.2123	0.6416	35.43	18.46	2.37e-04	2.66
2500	2.28	0.1885	0.6682	49.61	34.99	2.28e-04	2.33
2750	2.50	0.1734	0.6513	31.96	15.30	2.19e-04	2.27
3000	2.73	0.1532	0.6563	31.40	15.44	2.10e-04	2.93
3250	2.96	0.1287	0.6604	48.16	34.82	2.00e-04	1.70
3500	3.19	0.1061	0.6573	44.59	30.61	1.91e-04	1.50
3750	3.42	0.0947	0.6516	34.12	17.72	1.82e-04	1.65
4000	3.64	0.0940	0.6546	29.34	13.25	1.73e-04	1.65
4250	3.87	0.0791	0.6389	32.56	16.25	1.64e-04	1.77
4500	4.10	0.0649	0.6551	37.08	21.99	1.55e-04	1.24
4750	4.33	0.0573	0.6678	33.99	19.43	1.46e-04	0.84
5000	4.55	0.0546	0.6536	27.46	12.05	1.37e-04	1.02
5250	4.78	0.0469	0.6603	28.65	13.21	1.28e-04	1.11
5500	5.01	0.0390	0.6679	29.75	15.28	1.19e-04	0.97
5750	5.24	0.0315	0.6669	28.00	13.77	1.10e-04	0.53
6000	5.46	0.0270	0.6814	32.93	17.82	1.01e-04	0.68
6250	5.69	0.0236	0.6527	29.96	16.07	9.18e-05	0.50
6500	5.92	0.0220	0.6690	23.19	9.03	8.27e-05	0.70
6750	6.15	0.0166	0.6889	25.64	11.19	7.37e-05	0.90
7000	6.38	0.0125	0.6636	27.15	12.85	6.46e-05	0.51
7250	6.60	0.0097	0.6960	29.56	15.38	5.56e-05	0.44
7500	6.83	0.0101	0.6943	30.11	16.66	4.65e-05	0.65
7750	7.06	0.0076	0.6850	27.00	13.23	3.75e-05	0.82
8000	7.29	0.0072	0.6907	25.29	11.81	2.84e-05	0.10
8250	7.51	0.0048	0.6933	27.07	13.31	1.94e-05	0.05
8500	7.74	0.0044	0.6936	25.82	12.25	1.03e-05	0.26
8750	7.97	0.0046	0.6891	25.88	12.37	1.27e-06	0.24