This model transcribes speech in lowercase Basque alphabet including spaces, and was trained on a composite dataset comprising of 548 hours of Basque speech. The model was fine-tuned from a pre-trained Spanish stt_es_conformer_ctc_large model using the Nvidia NeMo toolkit. It is a non-autoregressive "large" variant of Conformer, with around 121 million parameters.
See the model architecture section and NeMo documentation for complete architecture details.
Usage
To train, fine-tune or play with the model you will need to install NVIDIA NeMo. We recommend you install it after you've installed latest PyTorch version.
Given NEMO_MODEL_FILEPATH is the path that points to the downloaded stt_eu_conformer_ctc_large.nemo file.
python
1import nemo.collections.asr as nemo_asr
23# Load the model4asr_model = nemo_asr.models.EncDecCTCModelBPE.restore_from(NEMO_MODEL_FILEPATH)56# Create a list pointing to the audio files7audio =["audio_1.wav","audio_2.wav",...,"audio_n.wav"]89# Fix the batch_size to whatever number suits your purpouse10batch_size =81112# Transcribe the audio files13transcriptions = asr_model.transcribe(audio=audio, batch_size=batch_size)1415# Visualize the transcriptions16print(transcriptions)
Change decoding strategy
Optionally you can add some lines before transcribing the audio to change the decoding strategy and use Beam Search with N-gram Language Model. The previous installation of the beam search decoders has been made using the script provided by the NeMo Toolkit [3]. Given KENLM_MODEL_FILEPATH is the path that points to the downloaded kenlm_unigram_v256_model.bin file.
This model accepts 16000 kHz Mono-channel Audio (wav files) as input.
Output
This model provides transcribed speech as a string for a given audio sample.
Model Architecture
Conformer-CTC model is a non-autoregressive variant of Conformer model [1] for Automatic Speech Recognition which uses CTC loss/decoding instead of Transducer. You may find more info on the detail of this model here: Conformer-CTC Model.
Training
Data preparation
This model has been trained on a composite dataset comprising 548 hours of Basque speech that contains:
A processed subset of the validated split of the basque version of the public dataset Mozilla Common Voice 16.1: We have processed the validated split, which originally contains the train, dev and test splits, to create a subset free of sentences equal to the ones that are in the test split, to avoid leakage.
The train_clean split of the basque version of the public dataset Basque Parliament
A processed subset of the basque version of the public dataset OpenSLR: This subset has been cleaned from numerical characters and acronyms.
The composite dataset for training has been precisely cleaned from any sentence that equals the ones in the test datasets where the WER metrics will be computed.
Training procedure
This model was trained starting from the pre-trained Spanish model stt_es_conformer_ctc_large over several hundred of epochs in a GPU device, using the NeMo toolkit [3]
The tokenizer for these model was built using the text transcripts of the composite train dataset with this script, with a total of 256 basque language tokens.
Performance
Performances of the ASR models are reported in terms of Word Error Rate (WER%) with greedy decoding in the following table.
Tokenizer
Vocabulary Size
MCV 16.1 Test
Basque Parliament Test
Basque Parliament Dev
Train Dataset
SentencePiece Unigram
256
4.72
4.51
4.85
Composite Dataset (548 h)
A N-gram Language model has been trained using the script provided in the NeMo Toolkit [3] with a corpus comprissed of 27 million basque language sentences from accesible open sources like:
Tatoeba, OpenSubtitles, TED, GlobalVoices, and other corpora from OPUS
Performances of the ASR models are reported in terms of Word Error Rate (WER%) with beam-search decoding with N-gram LM in the following table.
N
Beam Size
Beam Alpha
Beam Beta
MCV 16.1 Test
Basque Parliament Test
Basque Parliament Dev
6
32
1
1
2.42
4.21
4.3
Limitations
Since this model was trained on publicly available speech datasets, the performance of this model might degrade for speech which includes technical terms, or vernacular that the model has not been trained on. The model might also perform worse for accented speech.
Aditional Information
Author
HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.
Licensing Information
Copyright (c) 2024 HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Funding
This project with reference 2022/TL22/00215335 has been parcially funded by the Ministerio de Transformación Digital and by the Plan de Recuperación, Transformación y Resiliencia – Funded by the European Union – NextGenerationEU ILENIA and by the project IkerGaitu funded by the Basque Government.
This model was trained at Hyperion, one of the high-performance computing (HPC) systems hosted by the DIPC Supercomputing Center.
Click to expand
The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
In no event shall the owner and creator of the models (HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.) be liable for any results arising from the use made by third parties of these models.
