Wolof-HuBERT is continued pretrained from
facebook/hubert-large-ls960 on 860 hours of Wolof speech audio. When using the model make sure that your speech input is also sampled at 16Khz.
Note: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model for
speech recognition, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out
huggingface transformers examples to finetune this model.
We experimented with finetuning, and the model outperforms other models of the same size.
See
this blog for more information on how to fine-tune the model. Note that the class
Wav2Vec2ForCTC has to be replaced by
HubertForCTC.
1@misc{sy2025speechlanguagemodelsunderrepresented,
2 title={Speech Language Models for Under-Represented Languages: Insights from Wolof},
3 author={Yaya Sy and Dioula Doucouré and Christophe Cerisara and Irina Illina},
4 year={2025},
5 eprint={2509.15362},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2509.15362},
9}
