Table of Contents
Click to expand
Model Description
This is a HuBERT Base model pre-trained using 6,000 hours of Iberian languages speech data (Spanish, Catalan, Basque, Galician, Aranese, plus English).
The model architecture is the same as the
original HuBERT Base model, which contains 12 transformer layers.
Pre-training was done by
Barcelona Supercomputing Center.
Intended Uses and Limitations
This pre-trained model generates Speech Representations that can be used for any Iberian speech-related task.
This model does not have a tokenizer as it was pretrained on audio alone.
In order to use this model for Automatic Speech Recognition, a tokenizer should be created and the model should be fine-tuned on labeled text data.
Check out
this blog for more in-detail explanation of how to fine-tune the model for Speech Recognition.
For an explanation of how to fine-tune the model for Audio Classification, check out
this tutorial.
Pre-training Details
This model was pre-trained using code from the
official repository, and the detailed training configuration can be found in the same repository and the
original paper.
For pre-training, a 6,000 hours dataset was created using subsets from training splits from the following datasets:
Indirect evaluation results
To assess the pre-trained Speech Representations' quality, we evaluated them using two indirect tasks: Automatic Speech Recognition (ASR) and Language Identification (LID).
Automatic Speech Recognition
For each language (Spanish, Catalan, Basque, Galician, Aranese, and English), we created train and validation ASR-labelled datasets using 50 hours subsamples from Common Voice train splits (except for Aranese, were we used 2 hours of Aranese, plus 30 hours of Catalan).
For testing, we used each language Common Voice test split.
We fine-tuned on this ASR-labelled training splits the following models:
- Iberian pre-trained HuBERT: BSC-LT/hubert-base-los-6k (our new model)
- Iberian pre-trained HuBERT: BSC-LT/hubert-base-los-2k (our previous model)
- Multi-lingual pre-trained HuBERT: utter-project/mHuBERT-147
All of these models were pre-trained using exactly the same configurations.
We trained them for 20 epochs.
For the fine-tuning process, we froze models' parameters using the freeze_feature_encoder() method.
All models have 94M parameters, 95% of them were fine-tuned.
The results were the following:
| Model | Avg WER ↑ | Spanish WER | Catalan WER | Basque WER | Galician WER | Aranese WER | English WER |
|---|
| hubert-base-los-6k | 22.4% | 17.3% | 17.6% | 10.6% | 11.4% | 31.4% | 46.0% |
| mHuBERT-147 | 24.7% | 21.4% | 22.6% | 15.3% | 15.8% | 38.7% | 34.6% |
| hubert-base-los-2k | 25.7% | 16.7% | 17.5% | 9.1% | 11.4% | 44.3% | 55.5% |
Language Identification
We created train and validation Language Identification labelled splits using subsamples from each language Common Voice train split.
That is, we randomly selected 5 hours for Spanish, 5 hours for Catalan, 5 hours for Basque, 5 hours for Galician, 5 hours for English and 2 hours for Aranese.
For testing, we created a test split concatenating all the Spanish, Catalan, Basque, Galician, Aranese and English test splits from Common Voice.
We fine-tuned on this 22 hours labelled training split the following models:
- Iberian pre-trained HuBERT: BSC-LT/hubert-base-los-6k (our new model)
- Iberian pre-trained HuBERT: BSC-LT/hubert-base-los-2k (our previous model)
- Multi-lingual pre-trained HuBERT: utter-project/mHuBERT-147
All of these models were pre-trained using exactly the same configurations.
We trained them for 10 epochs.
For the fine-tuning process, we froze models' parameters using the freeze_base_model() method.
All the models have 94M parameters, 0.2% of them were fine-tuned.
The results were the following:
| Model | F1-score Macro ↓ | Spanish F1-score | Catalan F1-score | Basque F1-score | Galician F1-score | Aranese F1-score | English F1-score |
|---|
| hubert-base-los-6k | 80.5% | 91.2% | 90.4% | 98.8% | 90.3% | 15.0% | 97.5% |
| hubert-base-los-2k | 80.3% | 92.2% | 90.2% | 98.9% | 94.1% | 12.9% | 93.8% |
| mHuBERT-147 | 72.3% | 82.7% | 83.5% | 91.1% | 75.2% | 7.3% | 93.9% |
How to use the model
Speech Representations
To obtain Speech Representations (HuBERT outputs) from audio in Iberian languages using this model, you can follow this example:
(Using fsspec==2025.3.0, datasets==3.6.0 and transformers==4.52.2 is recomended).
1
2from datasets import load_dataset, Audio
3import torch
4from transformers import AutoFeatureExtractor, AutoModel
5
6#Load the dataset
7dataset = load_dataset("projecte-aina/ib3_ca_asr", split='train[:1%]', trust_remote_code=True)
8
9#Downsample to 16kHz
10dataset = dataset.cast_column("audio", Audio(sampling_rate=16_000))
11
12# Hugginface pre-trained model path
13MODEL_NAME = "BSC-LT/hubert-base-los-6k"
14
15# Set device
16device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
17print(f"Using {device} device.")
18
19# Load feature extractor
20feature_extractor = AutoFeatureExtractor.from_pretrained(MODEL_NAME)
21
22# Load model
23model = AutoModel.from_pretrained(MODEL_NAME)
24model = model.to(device)
25
26def map_to_speech_representations(batch):
27
28 #Process the dataset
29 audio = batch["audio"]
30 input_features = feature_extractor(audio["array"], sampling_rate=audio["sampling_rate"], return_tensors="pt").input_values
31 input_features = input_features.to(device)
32
33 # Extract HuBERT's Speech Representations
34 with torch.no_grad():
35 outputs = model(
36 input_features,
37 output_hidden_states = True,
38 )
39 speech_representations = outputs.last_hidden_state
40 hidden_states = outputs.hidden_states
41
42 batch["speech_representations"] = speech_representations
43 batch["hidden_states"] = hidden_states
44
45 return batch
46
47dataset = dataset.map(map_to_speech_representations)
48
49print(dataset)
Discrete Speech Representations
Important remark: the k-means model available in this repo and used for extracting Discrete Speech Representations was trained using HuBERT's 6th layer.
To obtain Discrete Speech Representations (HuBERT's k-means centroids) from audio in Iberian languages using this model, you can follow this example:
(Using fsspec==2025.3.0, datasets==3.6.0 and transformers==4.52.2 is recomended).
1
2from datasets import load_dataset, Audio
3import torch
4from transformers import AutoFeatureExtractor, AutoModel
5import joblib
6import numpy as np
7from huggingface_hub import hf_hub_download
8
9#Load the dataset
10dataset = load_dataset("projecte-aina/ib3_ca_asr", split='train[:1%]', trust_remote_code=True)
11
12#Downsample to 16kHz
13dataset = dataset.cast_column("audio", Audio(sampling_rate=16_000))
14
15# Hugginface pre-trained model path
16MODEL_NAME = "BSC-LT/hubert-base-los-6k"
17
18# Set device
19device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
20print(f"Using {device} device.")
21
22# Load feature extractor
23feature_extractor = AutoFeatureExtractor.from_pretrained(MODEL_NAME)
24
25# Load model
26model = AutoModel.from_pretrained(MODEL_NAME)
27model = model.to(device)
28
29# Load k-means
30km_path = hf_hub_download(repo_id="BSC-LT/hubert-base-los-2k", filename="k_means.km")
31km_model = joblib.load(km_path)
32clusters = km_model.cluster_centers_
33
34def map_to_discrete_units(batch):
35
36 #Process the dataset
37 audio = batch["audio"]
38 input_features = feature_extractor(audio["array"], sampling_rate=audio["sampling_rate"], return_tensors="pt").input_values
39 input_features = input_features.to(device)
40
41
42 with torch.no_grad():
43 outputs = model(
44 input_features,
45 output_hidden_states = True,
46 )
47
48 # Extract HuBERT's Speech Representations
49 hidden_states = outputs.hidden_states
50
51 # Extract 6-th layer features
52 k_means_input = hidden_states[5].squeeze()
53 k_means_input = k_means_input.cpu()
54 k_means_input = np.array(k_means_input, dtype='f')
55
56 labels = km_model.predict(k_means_input)
57 batch["discrete_units"] = clusters[labels]
58
59 return batch
60
61dataset = dataset.map(map_to_discrete_units)
62
63print(dataset)
64
Automatic Speech Recognition
In order to use this model for Speech Recognition, a tokenizer should be created and the model should be fine-tuned on labeled text data.
Check out
this blog for more in-detail explanation of how to fine-tune the model for Speech Recognition.
Audio Classification
For an explanation of how to fine-tune the model for Audio Classification, check out
this tutorial.
Citation
If this model contributes to your research, please cite the work:
1@misc{costa2026hubertbaselos6k,
2 title={LOSHuBERT: the first full Iberian pre-trained HuBERT.},
3 author={Costa, Federico; Messaoudi, Abir; Peiró-Lilja, Alex; Casals-Salvador, Marc; España-Bonet, Cristina},
4 organization={Barcelona Supercomputing Center},
5 url={https://huggingface.co/BSC-LT/hubert-base-los-6k},
6 year={2026}
7}
Additional Information
Author
Contact
For further information, please send an email to
bsc-lt@bsc.es.
Copyright
Copyright(c) 2026 by AI Institute, Barcelona Supercomputing Center.
License
Funding
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA.
The training of the model was possible thanks to the computing time provided by
Barcelona Supercomputing Center through MareNostrum 5.
We acknowledge EuroHPC Joint Undertaking for awarding us access to MareNostrum5 as BSC, Spain.
Acknowledgements
We acknowledge the following collaborations:
- Proxecto Nós (Instituto da Lingua Galega, USC), for developing Nos_RG-Podcast-GL, Nos_ParlaSpeech-GL, Nos_Transcrispeech-GL, Nos_Celtia-GL datasets.
- The Corts Valencianes data have been collected thanks to the intervention of the NEL-VIVES campaign, an initiative developed by Cenid, the Digital Intelligence Center of the University of Alicante.
