Whisper Large Spanish is a high-accuracy automatic speech recognition (ASR) model for Spanish (es), fine-tuned from [openai/whisper-large] on the Spanish subset of Mozilla Common Voice 13.0. It achieves a Word Error Rate (WER) of 5.1265% on the evaluation set.

This model is designed for applications that require near state-of-the-art transcription accuracy in Spanish, such as transcription of lectures, podcasts, and other high-quality recordings.

Model description

Architecture: Transformer-based encoder–decoder (Whisper Large)
Base model: openai/whisper-large
Language: Spanish (es)
Task: Automatic Speech Recognition (ASR)
Output: Text transcription in Spanish
Decoding: Autoregressive sequence-to-sequence decoding

Large model offers very high accuracy at the cost of higher computational requirements compared to Medium or Small variants.

Intended use

Primary use cases

High-accuracy Spanish speech transcription
Applications requiring transcription of long-form audio
Research in Spanish ASR performance and benchmarking

Limitations

May underperform in extremely noisy audio or with strong regional accents not well represented in the Common Voice dataset
High computational cost for real-time inference
Not suitable for legal, medical, or safety-critical applications without human review

Training and evaluation data

Dataset: Mozilla Common Voice 13.0 (Spanish subset)
Data type: Crowd-sourced read speech
Preprocessing:
- Audio resampled to 16 kHz
- Text tokenized using Whisper tokenizer
- Removal of invalid or corrupted samples
Evaluation metric: Word Error Rate (WER) on held-out evaluation set

Evaluation results

Metric	Value
WER (eval)	5.1265%

Training procedure

Training hyperparameters

Learning rate: 1e-5
Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
LR scheduler: Linear
Warmup steps: 500
Training steps: 20000
Train batch size: 32 (gradient accumulation 2 → effective batch size 64)
Eval batch size: 16
Seed: 42

Training results (summary)

Training Loss	Epoch	Step	Validation Loss	WER
0.0834	2.0	1000	0.1862	6.3852
0.0871	4.0	2000	0.1777	5.9175
0.039	6.0	3000	0.1780	5.7423
0.0265	8.0	4000	0.2121	5.7744
0.0059	10.0	5000	0.2219	5.8097
0.0855	12.01	6000	0.1839	5.9778
0.0037	14.01	7000	0.2273	5.8565
0.0293	16.01	8000	0.1965	5.8078
0.1174	18.01	9000	0.1984	5.8893
0.0355	20.01	10000	0.2136	5.8662
0.0279	22.01	11000	0.1882	5.4960
0.0043	24.01	12000	0.2444	5.3356
0.0302	26.01	13000	0.2223	5.4620
0.0011	28.01	14000	0.2603	5.5608
0.001	30.01	15000	0.2452	5.3087
0.0003	32.01	16000	0.2573	5.3523
0.0004	34.02	17000	0.2690	5.2952
0.0013	36.02	18000	0.2373	5.1438
0.0004	38.02	19000	0.2618	5.1361
0.0004	40.02	20000	0.2663	5.1265

Framework versions

Transformers 4.33.0.dev0
PyTorch 2.0.1+cu117
Datasets 2.14.4
Tokenizers 0.13.3

Example usage

python

from transformers import pipeline

hf_model = "HiTZ/whisper-large-es"  # replace with actual repo ID
device = 0  # -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

This model transcribes speech and may process personal data.
Users should ensure compliance with applicable data protection laws (e.g., GDPR).
The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

bibtex

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

Contact and attribution

Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
Base model: OpenAI Whisper
Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.