This model is a StyleTTS2 version trained from scratch in Spanish. The text aligner and the prosodic text encoder have also been specifically trained for Spanish.

StyleTTS2 is an end-to-end expressive text-to-speech (TTS) model that generates the most suitable style for the text without requiring reference speech. The model leverages style diffusion and adversarial training with large Speech Language Models (SLMs) to achieve human-level TTS synthesis. It uses an end-to-end (E2E) training process that jointly optimizes all components, along with direct waveform synthesis and adversarial training with large SLMs enabled by differentiable duration modeling. The speech style is modeled as a latent variable sampled through diffusion models, allowing diverse speech generation without reference audio.

This model is intended to serve as multispeaker text-to-speech system for the Spanish language. It has been trained using a Spanish phonemizer, therefore if the model is used for other languages it will not produce intelligible samples. The quality of the samples can vary depending on the quality and length of the audio prompt.

CML-TTS open source dataset open source dataset have been used as training and validation data. The data has been filtered to include only audios up to 20 seconds, which corresponds to a maximum length of 800 frames. The total duration of the dataset is 7.9 hours for training and 0.2 hours for validation. All samples were resampled to 24kHz.

The model in this repository (styletts2-spanish-multispeaker) was trained from scratch. Two other models (FT-libri-tts_asr-es_pl-bert-subword-es_wavlm-base-plus and FT-kokoro_asr-es_pl-bert-subword-es_wavlm-base-plus) were fine-tuned for comparison and evaluation. The original pitch extractor was used for all models. A Spanish AuxiliaryASR was trained as text aligner and used for all models. BSC-LT/PL-BERT-wp-es was used as prosodic text Encoder for all models. microsoft/wavlm-base-plus was used as SLM for all models.

styletts2-spanish-multispeaker was trained from scratch. The first stage was executed for 99 epochs. The second stage was executed for 86 epochs.

FT-libri-tts_asr-es_pl-bert-subword-es_wavlm-base-plus was fine-tuned and yl4579/StyleTTS2-LibriTTS was used as pre-trained model. The fine-tuning stage was executed for 32 epochs.

FT-kokoro_asr-es_pl-bert-subword-es_wavlm-base-plus was fine-tuned and hexgrad/Kokoro-82M was used as pre-trained model. The fine-tuning stage was executed for 79 epochs.

styletts2-spanish-multispeaker first stage hyperparameters can be found here and second stage hyperparameters here.

FT-libri-tts_asr-es_pl-bert-subword-es_wavlm-base-plus fine-tuning hyperparameters can be found here.

FT-kokoro_asr-es_pl-bert-subword-es_wavlm-base-plus fine-tuning hyperparameters can be found here.

The test corpus is comprised of 300 text sentences from "Niebla" by Miguel de Unamuno. The text was sourced from Project Gutenberg’s website. The sentences can be found here. The distribution of the sentences is the following:

We used 4 speakers (2 males and 2 females) to generate the audio samples.

A mix of automatic and subjective metrics has been used to evaluate this model. For the automatic metrics we used VERSA library. UTMOS and DNSMOS Pro BVCCC where used to assess the audio quality and WER was used to assess the intelligibility of the models.

A total of 3 models has been evaluated with 3 automatic metrics (UTMOS, DNSMOS and WER). The model in this repository (styletts2-spanish-multispeaker) has been compared to a finetuning:

If this model contributes to your research, please cite the work:

The Speech team of the Barcelona Supercomputing Center.

For further information, please send an email to bsc-lt@bsc.es.

Copyright(c) 2026 by AI Institute, Barcelona Supercomputing Center.

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA. The training of the model was possible thanks to the computing time provided by Barcelona Supercomputing Center through MareNostrum 5. We acknowledge EuroHPC Joint Undertaking for awarding us access to MareNostrum5 as BSC, Spain.