OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech with superior inference speed, supporting voice cloning and voice design.
600+ Languages Supported: The broadest language coverage among zero-shot TTS models.
Voice Cloning: State-of-the-art voice cloning quality from a short reference audio.
Voice Design: Control voices via assigned speaker attributes (gender, age, pitch, dialect/accent, whisper, etc.).
Fine-grained Control: Non-verbal symbols (e.g., [laughter]) and pronunciation correction via pinyin or phonemes.
Fast Inference: RTF as low as 0.025 (40x faster than real-time).
Diffusion Language Model-style Architecture: A clean, streamlined, and scalable design that delivers both quality and speed.
Usage
To get started, install the omnivoice library:
We recommend using a fresh virtual environment (e.g., conda, venv, etc.) to avoid conflicts.
Step 1: Install PyTorch
NVIDIA GPU
bash
1# Install pytorch with your CUDA version, e.g.2pip installtorch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128
You can use OmniVoice for zero-shot voice cloning as follows:
python
1from omnivoice import OmniVoice
2import soundfile as sf
3import torch
45# Load the model6model = OmniVoice.from_pretrained(7"k2-fsa/OmniVoice",8 device_map="cuda:0",9 dtype=torch.float16
10)1112# Generate audio13audio = model.generate(14 text="Hello, this is a test of zero-shot voice cloning.",15 ref_audio="ref.wav",16 ref_text="Transcription of the reference audio.",17)# audio is a list of `np.ndarray` with shape (T,) at 24 kHz.1819sf.write("out.wav", audio[0],24000)
For more generation modes (e.g., voice design), functions (e.g., non-verbal symbols, pronunciation correction) and comprehensive usage instructions, see our GitHub Repository.
You can also scan the QR code to join our wechat group or follow our wechat official account.
Wechat Group
Wechat Official Account
wechat
wechat
Citation
bibtex
1@article{zhu2026omnivoice,
2 title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
3 author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
4 journal={arXiv preprint arXiv:2604.00688},
5 year={2026}
6}
License
Our code is released under the Apache 2.0 License. The pre-trained model is licensed under the CC-BY-NC due to constraints from its training data (e.g., Emilia).
Disclaimer
Users are strictly prohibited from using this model for unauthorized voice cloning, voice impersonation, fraud, scams, or any other illegal or unethical activities. All users shall ensure full compliance with applicable local laws, regulations, and ethical standards. The developers assume no liability for any misuse of this model and advocate for responsible AI development and use, encouraging the community to uphold safety and ethical principles in AI research and applications.
