TTS_T5-uz_ru_en is a multilingual text-to-speech (TTS) synthesis system built upon a modified encoder-decoder T5 architecture template. Developed using the orchestration engine in the companion firdavsus/T5-speech GitHub repository.

The framework provides robust cross-lingual acoustic stability across three target languages: Uzbek (uz), Russian (ru), and English (en).

voice-clonning (UZ, RU, EN)

Base voice -> Listen to the Audio Track

Model Features & Specifications

Task: Text-to-Speech / Text-to-Audio Synthesis
Core Engine: Sequence-to-sequence T5 (Text-to-Text Transfer Transformer) adaptation configured to map text tokens to target speech representations.
Languages Supported: Uzbek, Russian, and English.
Output Profiles: Compatible with neural vocoders (e.g., HiFi-GAN) or autoregressive audio token decoders to synthesize raw 16kHz/24kHz waveforms.

Architectural Workflow

The model re-imagines speech synthesis as a standard sequence-to-sequence translation task:

Phoneme/Text Encoding: The T5 text encoder processes character sequences, expanding them with cross-lingual language tags to preserve regional accents.
Acoustic Decoding: The autoregressive decoder generates sequential acoustic frames (such as mel-spectrogram slices) or discrete codebook indices conditioned on the encoder's hidden representations.
Vocoding Stage: Generated representations are passed to a neural vocoder block to output natural-sounding human audio speech.

Intended Uses & Limitations

Target Applications

Multilingual Assistant Platforms: Powering interactive voice agents or accessibility tools requiring fluent localized speech in Uzbekistan.
Speech Synthesis Research: Investigating text-to-speech capabilities within encoder-decoder LLM frameworks without separating the acoustic and text pipelines.
Audio Tokenization: Creating down-stream audio tokens suitable for multimodal or speech-to-speech language modeling.

Limitations

Prosody Control: Fine-grained pacing variations (e.g., intense whispering or shouting) may require explicit token prefixing depending on the configuration layer chosen during training.
Vocoder Dependency: The raw audio fidelity depends directly on the vocoder setup applied to decode the model's structural audio predictions.

Quickstart Inference

You can run sequence generation and audio rendering loops using the evaluation pipeline scripts available in the primary GitHub repository.

python

import torch
from model import T5SpeechGenerator  # Imported from your firdavsus/T5-speech repository
import scipy.io.wavfile as wavfile

# 1. Initialize the multilingual TTS pipeline
device = "cuda" if torch.cuda.is_available() else "cpu"
model = T5SpeechGenerator.from_pretrained("firdavsus/TTS_T5-uz_ru_en").to(device)

# 2. Prepare text input with language formatting prefix
text_prompt = "<uz> Assalomu alaykum! Bugun ob-havo juda yaxshi."

# 3. Generate raw audio waveforms
with torch.no_grad():
    audio_outputs = model.generate_speech(text_prompt)

# 4. Save synthesized audio track to local disk
wavfile.write("output_uzbek.wav", rate=24000, data=audio_outputs.cpu().numpy())
print("Multilingual speech successfully synthesized and exported to disk.")