Whisper-Small Dutch - Common Voice Only (Baseline)
This model is a fine-tuned version of openai/whisper-small for Dutch automatic speech recognition (ASR). It was trained exclusively on Common Voice 17.0 Dutch without any synthetic data augmentation, serving as a baseline for comparison with synthetic-augmented models.
Introduction
Purpose
This model serves as the baseline for evaluating the effectiveness of synthetic data augmentation in Dutch ASR. By training only on real speech data from Common Voice 17.0, we establish reference performance metrics against which synthetic-augmented models can be compared.
How the Model Was Created
The model was fine-tuned from openai/whisper-small using the Hugging Face Transformers library:
Training Data: 34,952 real speech samples from Common Voice 17.0 Dutch (train split).
Optimization: Trained for 5 epochs with a learning rate of 1e-5, global batch size of 256, and BF16 precision on an NVIDIA H200 GPU.
Checkpoint Selection: The best checkpoint was selected based on validation loss, occurring at step 400 with a validation loss of 0.1492.
This baseline achieves 11.13% WER on the Common Voice test set, which synthetic-augmented models improve upon by up to 2.4% relative.
Model Details
Property
Value
Base Model
openai/whisper-small
Language
Dutch (nl)
Task
Automatic Speech Recognition (transcribe)
Parameters
244M
Training Data
Common Voice 17.0 Dutch only
Total Training Samples
34,952
Sampling Rate
16kHz
Evaluation Results
This Model (whisper-small-cv-only-nl)
Metric
Value
Validation Loss
0.1491
Validation WER
8.73%
Test WER (Common Voice)
11.13%
Test WER (MLS)
30.71%
Best Checkpoint
Step 400
Max Training Steps
680
Comparison with Synthetic-Augmented Models (Whisper-Small Dutch)
Training Data
Max Steps
Val Loss
Val WER
Test WER (CV)
Test WER (MLS)
Common Voice Only
680
0.1491
8.73%
11.13%
30.71%
High-Quality Filtered + CV
890
0.1493
8.76%
11.00%
29.91%
Mid-High Quality Filtered + CV
1,270
0.1484
8.73%
10.86%
30.04%
All Synthetic + CV (Unfiltered)
1,365
0.1484
8.64%
10.91%
30.06%
Key Observations
Baseline performance: 11.13% Test WER on Common Voice, 30.71% on MLS
Fastest training: Only 680 max steps (smallest dataset)
Room for improvement: Synthetic augmentation reduces Test WER by up to 0.27% absolute (2.4% relative)
Cross-domain gap: 19.58% absolute difference between CV and MLS performance highlights domain mismatch
Common Voice 17.0 Dutch contains crowdsourced voice recordings from volunteer contributors reading text prompts. The dataset provides diverse speaker demographics but is limited in acoustic conditions and speaking styles.
