This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

Wer: 19.7605
Cer: 3.7878
Real Time Factor: 0.0024

Quick Inference

python

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

E1 (Baseline): Standard training configuration.
E2 (Noise): Training with background noise augmentation.
E3 (Speed): Training with speed perturbation augmentation.
E4 (SpecAug): Training with SpecAugment (time and frequency masking).
E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-300m-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: OptimizerNames.ADAMW_TORCH_FUSED
lr_scheduler_type: SchedulerType.LINEAR
num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.91	10.9711	2.4033	99.72	91.88	1.49e-04	2.63
500	1.82	3.1841	0.4280	45.31	11.44	2.99e-04	2.52
750	2.73	2.3094	0.2775	32.74	7.43	2.56e-04	3.22
1000	3.64	1.8766	0.2251	27.46	5.96	2.12e-04	3.20
1250	4.55	1.4938	0.1916	23.92	5.01	1.68e-04	2.80
1500	5.46	1.2829	0.1783	22.01	4.54	1.24e-04	2.71
1750	6.36	1.2294	0.1607	20.72	4.14	7.96e-05	3.18
2000	7.27	1.1032	0.1517	18.84	3.80	3.55e-05	2.87
2200	8.00	1.0568	0.1485	18.31	3.66	1.76e-07	1.83