Model Card for Model ID: zikangzheng/speecht5-chinese-common_voice_17_0-zh-cn
This model fine-tunes Microsoft’s SpeechT5 for Mandarin Chinese text-to-speech (TTS) using the Common Voice 17.0 zh-CN dataset. It generates natural, clear Mandarin speech from input text.
Model Details
Model Description
This model adapts microsoft/speecht5_tts for Mandarin Chinese speech synthesis. It was trained on the Common Voice 17.0 (zh-CN) dataset with one epoch of fine-tuning on an NVIDIA RTX 4060 Laptop GPU under Windows.
The model maps Chinese text sequences to mel-spectrograms, which can be converted into audio using a HiFi-GAN vocoder. It is useful for research, accessibility, or educational purposes related to Mandarin speech synthesis.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: Zikang (Khan) Zheng
- Model type: Text-to-Speech (TTS)
- Language(s) (NLP): Chinese (zh-CN)
- License: CC BY 4.0 (dataset), MIT (code)
- Finetuned from model : microsoft/speecht5_tts
Model Sources [optional]
Uses
Direct Use
Mandarin text-to-speech synthesis for:
- Accessibility (e.g., screen readers)
- Language education tools
- Research in multilingual TTS systems
Downstream Use [optional]
- Fine-tuning for specific speaker voices or dialects
- Integration into larger voice interaction systems
Out-of-Scope Use
- Voice cloning or impersonation of real individuals
- Commercial deployment without further testing
Bias, Risks, and Limitations
The dataset (Common Voice) contains recordings from various speakers with different accents and microphone conditions.
As such:
- Audio quality may vary.
- Pronunciations for rare characters may be inconsistent.
- Tone accuracy may degrade for complex phrases.
Recommendations
Use high-quality vocoders (e.g., SpeechT5HiFiGAN) for best results.
Avoid using the model for personal voice replication or misleading content.
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech, SpeechT5HifiGan
import torch, soundfile as sf
processor = SpeechT5Processor.from_pretrained("zikangzheng/speecht5-chinese-common_voice_17_0-zh-cn")
model = SpeechT5ForTextToSpeech.from_pretrained("zikangzheng/speecht5-chinese-common_voice_17_0-zh-cn")
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
inputs = processor(text="你好，欢迎使用中文语音合成模型。", return_tensors="pt")
with torch.no_grad():
speech = model.generate_speech(inputs["input_ids"], vocoder=vocoder)
sf.write("output.wav", speech.numpy(), samplerate=16000)
Training Details
Training Data
Dataset: Mozilla Common Voice 17.0 zh-CN
Sampling Rate: 16 kHz
Subset Used: zh-CN_train_0
Preprocessing:
- Text normalization
- Resampling to 16 kHz
- Conversion via SpeechT5Processor
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime:
- Epochs: 1
- Batch size: 1
- Gradient accumulation: 3
- Learning rate: 3e-5
- Precision: FP32
- Max grad norm: 1.0
- Save steps: 200
- Logging steps: 50
Speeds, Sizes, Times [optional]
| Metric | Value |
|---|
| Train runtime | 3687.28 sec (~61.5 min) |
| Samples per second | 7.98 |
| Steps per second | 2.66 |
| Final training loss | 0.8552 |
| Epochs completed | 1 |
Evaluation
Testing Data, Factors & Metrics
Testing Data
Sampled from Common Voice zh-CN validation subset for listening evaluation.
Factors
Speech clarity, tonal accuracy, and pronunciation of multi-character words.
Metrics
- Qualitative listening evaluation (clarity, tone accuracy)
- Loss convergence (cross-entropy on mel-spectrograms)
Results
| Metric | Value |
|---|
| Training loss | 0.8552 |
| Audio quality | Clear and natural |
| Tone accuracy | Stable for standard Mandarin |
Summary
The fine-tuned model achieved good pronunciation and stable tones for short and medium-length Mandarin phrases.
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: NVIDIA RTX 4060 Laptop GPU
- Hours used: ~1.0
- Cloud Provider: Local (Windows 11)
- Compute Region: Singapore
- Carbon Emitted: < 0.5 kg CO₂e
Technical Specifications [optional]
Model Architecture and Objective
Transformer-based encoder-decoder model for text-to-mel-spectrogram generation.
Vocoder: HiFi-GAN for waveform synthesis.
Objective: Minimize MSE loss between predicted and target mel-spectrograms.
Compute Infrastructure
Hardware
- GPU: NVIDIA RTX 4060 Laptop
- RAM: 32 GB
- VRAM: 8 GB
Software
- Transformers: 4.44+
- Datasets: 3.0+
- PyTorch: 2.3+
- Python: 3.10
- Accelerate: 0.33+
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
@misc{zikangzheng2025speecht5zhcn,
author = {Zheng, Zikang (Khan)},
title = {SpeechT5 Chinese TTS (Common Voice 17.0 zh-CN)},
year = {2025},
publisher = {Hugging Face},
howpublished = {\url{
https://huggingface.co/zikangzheng/speecht5-chinese-common_voice_17_0-zh-cn}}
}
APA:
Zheng, Z. (2025). SpeechT5 Chinese TTS (Common Voice 17.0 zh-CN). Hugging Face.
https://huggingface.co/zikangzheng/speecht5-chinese-common_voice_17_0-zh-cnGlossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
Author: Zikang (Khan) Zheng
Location: Singapore
Hugging Face:
https://huggingface.co/zikangzheng
