This model is a fine-tuned version of openai/whisper-medium on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

Wer: 26.3473
Cer: 10.6790
Real Time Factor: 0.0546

Quick Inference

python

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

E1 (Baseline): Standard training configuration.
E2 (Noise): Training with background noise augmentation.
E3 (Speed): Training with speed perturbation augmentation.
E4 (SpecAug): Training with SpecAugment (time and frequency masking).
E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: OptimizerNames.ADAMW_TORCH_FUSED
lr_scheduler_type: SchedulerType.LINEAR
num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.30	0.5439	0.6491	46.75	21.78	1.49e-04	6.48
500	0.61	2.0455	1.4417	64.19	40.93	2.99e-04	9.23
750	0.91	0.6062	0.8026	79.31	68.75	2.88e-04	5.40
1000	1.21	0.4792	0.7684	49.83	27.45	2.75e-04	4.66
1250	1.52	0.3800	0.6988	43.12	23.67	2.63e-04	4.61
1500	1.82	0.2845	0.6733	32.34	13.24	2.51e-04	3.43
1750	2.12	0.2145	0.7059	37.75	17.47	2.38e-04	2.72
2000	2.43	0.1927	0.6659	50.14	36.39	2.26e-04	2.25
2250	2.73	0.1722	0.6448	36.63	18.20	2.14e-04	2.69
2500	3.03	0.1153	0.6828	44.36	29.13	2.02e-04	1.91
2750	3.34	0.1133	0.6471	33.38	15.92	1.89e-04	1.97
3000	3.64	0.0923	0.6713	43.23	28.74	1.77e-04	2.38
3250	3.94	0.0774	0.6387	30.29	13.36	1.65e-04	1.54
3500	4.25	0.0568	0.6538	34.51	18.21	1.52e-04	1.37
3750	4.55	0.0498	0.6446	27.02	11.38	1.40e-04	1.09
4000	4.85	0.0387	0.6233	38.90	26.54	1.28e-04	1.08
4250	5.16	0.0242	0.6635	27.20	12.89	1.15e-04	0.97
4500	5.46	0.0216	0.6543	37.99	25.78	1.03e-04	0.60
4750	5.77	0.0136	0.6579	24.79	10.17	9.08e-05	0.60
5000	6.07	0.0041	0.6616	25.60	11.76	7.84e-05	0.06
5250	6.37	0.0044	0.6697	28.33	14.31	6.61e-05	0.01
5500	6.68	0.0047	0.6433	31.88	18.58	5.38e-05	0.31
5750	6.98	0.0030	0.6796	27.01	13.44	4.15e-05	0.02
6000	7.28	0.0016	0.6911	27.43	14.06	2.92e-05	0.04
6250	7.59	0.0007	0.7047	26.63	13.25	1.69e-05	0.01
6500	7.89	0.0007	0.7151	26.25	12.78	4.58e-06	0.00