This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

Wer: 24.0306
Cer: 5.1391
Real Time Factor: 0.0042

Quick Inference

python

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-1b-mizonal3-E3-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-1b-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

E1 (Baseline): Standard training configuration.
E2 (Noise): Training with background noise augmentation.
E3 (Speed): Training with speed perturbation augmentation.
E4 (SpecAug): Training with SpecAugment (time and frequency masking).
E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-1b-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-1b-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-1b-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-1b-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-1b-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: OptimizerNames.ADAMW_TORCH_FUSED
lr_scheduler_type: SchedulerType.LINEAR
num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.30	0.7400	0.4306	44.95	11.22	1.49e-04	1.18
500	0.61	0.4544	0.3607	40.47	9.99	2.99e-04	0.77
750	0.91	0.3348	0.2852	32.36	7.69	2.88e-04	0.82
1000	1.21	0.2393	0.2309	26.78	5.99	2.75e-04	1.10
1250	1.52	0.1961	0.2153	25.04	5.56	2.63e-04	0.59
1500	1.82	0.1514	0.2291	24.72	5.57	2.51e-04	0.39
1750	2.12	0.1142	0.2020	21.92	4.66	2.38e-04	0.46
2000	2.43	0.1140	0.2130	22.41	4.89	2.26e-04	0.31
2250	2.73	0.1346	0.2073	22.78	4.90	2.14e-04	0.54
2500	3.03	0.2568	0.3712	25.74	5.77	2.02e-04	0.83
2750	3.34	1.1509	1.2241	68.30	24.99	1.89e-04	3.00
3000	3.64	1.5828	1.4634	94.28	62.98	1.77e-04	1.01
3250	3.94	1.6651	1.5738	99.80	78.44	1.65e-04	1.55
3500	4.25	1.5348	1.4266	97.69	54.04	1.52e-04	2.07
3750	4.55	1.4105	1.3147	87.25	38.78	1.40e-04	1.71
4000	4.85	1.4030	1.3064	72.53	24.47	1.28e-04	3.74
4250	5.16	1.5054	1.4667	69.57	20.30	1.15e-04	5.71
4500	5.46	1.7501	1.6419	71.23	17.13	1.03e-04	3.50
4750	5.77	1.7955	1.6913	93.22	21.60	9.08e-05	0.00
5000	6.07	1.8022	1.6813	99.23	25.38	7.84e-05	0.00
5250	6.37	1.8675	1.6999	99.36	25.29	6.61e-05	0.00
5500	6.68	1.8234	1.6999	99.36	25.29	5.38e-05	0.00
5750	6.98	1.8892	1.6999	99.36	25.29	4.15e-05	0.00
6000	7.28	1.9035	1.6999	99.36	25.29	2.92e-05	0.00
6250	7.59	1.8068	1.6999	99.36	25.29	1.69e-05	0.00
6500	7.89	1.9037	1.6999	99.36	25.29	4.58e-06	0.00