Epikwhale/
speecht5_finetuned_telugu_charan

Name: speecht5_finetuned_telugu_charan
Availability: InStock

Are you the author?

GPU deploy isn't supported for this model type.

SpeechT5 Fine-tuned for Telugu Text-to-Speech

This model is a fine-tuned version of microsoft/speecht5_tts adapted for Telugu text-to-speech synthesis. The model was trained on SPRINGLab's IndicTTS Telugu dataset using a custom Telugu-to-Latin transliteration layer so the English-trained SpeechT5 tokenizer could process Telugu input.

Approach

SpeechT5's tokenizer is trained on English/Latin script and does not natively handle Devanagari-family scripts. Rather than retraining the tokenizer, this model uses a transliteration shim: Telugu input text is converted to Latin-script approximations before being passed to the model. The model then learns to map those transliterated sequences to Telugu speech during fine-tuning.

This is a pragmatic approach — it sidesteps the tokenizer problem but inherits any information loss from transliteration. It's not a substitute for native script support or phoneme-aware modeling.

Intended Uses

Telugu text-to-speech experimentation and research
Educational and accessibility prototypes for Telugu content
Baseline for further fine-tuning on richer Telugu datasets

Limitations

No formal speech-quality evaluation. This model was evaluated by training/validation loss only. There is no MOS study, no objective audio metric (MCD, PESQ, etc.), and no listener test. Quality should be judged by listening to outputs.
Latin-transliteration intermediate step. Some Telugu phonemic distinctions may not survive transliteration cleanly.
Single-speaker / dataset-bound. Performance is bounded by the IndicTTS Telugu dataset's speaker and content diversity.
No handling of code-mixed or technical text. English words, numerals, or mixed-script input are not handled robustly.
Compute-bound inference. Standard SpeechT5 inference requirements apply.

Training Data

SPRINGLab IndicTTS Telugu — 8,576 Telugu audio-text pairs. Used as-is, without additional filtering or augmentation.

Training Procedure

Hyperparameters

Learning rate: 1e-3
Train batch size: 4 (effective 32 with gradient accumulation = 8)
Eval batch size: 2
Optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
Scheduler: Linear with 100 warmup steps
Total steps: 1,000
Mixed precision: Native AMP
Seed: 42

Training Results

Validation loss decreased from 0.6689 → 0.4496 over 1,000 steps:

Step	Train Loss	Val Loss
100	0.7785	0.6689
200	0.8247	0.7610
300	0.6961	0.6406
400	0.6305	0.5726
500	0.5784	0.5422
600	0.5582	0.5184
700	0.5399	0.4992
800	0.5132	0.4786
900	0.4903	0.4617
1000	0.4774	0.4496

Loss-based convergence only — see limitations.

Usage

python

from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech
import torch

processor = SpeechT5Processor.from_pretrained("Epikwhale/speecht5_finetuned_telugu_charan")
model = SpeechT5ForTextToSpeech.from_pretrained("Epikwhale/speecht5_finetuned_telugu_charan")

# Telugu text must be transliterated to Latin script before tokenization.
# See the transliteration utility in the repo for the mapping used during training.
text = "your transliterated telugu text here"
inputs = processor(text=text, return_tensors="pt")

with torch.no_grad():
    speech = model.generate_speech(inputs["input_ids"], speaker_embeddings=None)

Speaker embeddings (e.g., SpeechBrain x-vectors) and a vocoder (e.g., HiFi-GAN) are required for the full inference pipeline.

Framework Versions

Transformers 4.47.0
PyTorch 2.5.1+cu121
Datasets 3.3.1
Tokenizers 0.21.0

Citation

bibtex

@misc{speecht5_telugu_charan,
  title  = {SpeechT5 Fine-tuned for Telugu Text-to-Speech},
  author = {Rama Charan Pisupati},
  year   = {2025},
  url    = {https://huggingface.co/Epikwhale/speecht5_finetuned_telugu_charan}
}

Acknowledgments

Microsoft Research — SpeechT5 architecture
SPRINGLab, IIT Madras — IndicTTS Telugu dataset
Hugging Face — transformers library and hosting

Contact

Email: rama.charan.official@gmail.com
GitHub: Epik-Whale463