Views
No views yet
[laughter]) and pronunciation correction via pinyin or phonemes.omnivoice library:We recommend using a fresh virtual environment (e.g.,conda,venv, etc.) to avoid conflicts.
1# Install pytorch with your CUDA version, e.g.
2pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128See PyTorch official site for other versions installation.
pip install torch==2.8.0 torchaudio==2.8.0pip install omnivoice1from omnivoice import OmniVoice
2import soundfile as sf
3import torch
4
5# Load the model
6model = OmniVoice.from_pretrained(
7 "k2-fsa/OmniVoice",
8 device_map="cuda:0",
9 dtype=torch.float16
10)
11
12# Generate audio
13audio = model.generate(
14 text="Hello, this is a test of zero-shot voice cloning.",
15 ref_audio="ref.wav",
16 ref_text="Transcription of the reference audio.",
17) # audio is a list of `np.ndarray` with shape (T,) at 24 kHz.
18
19sf.write("out.wav", audio[0], 24000)| Wechat Group | Wechat Official Account |
|---|---|
 |  |
1@article{zhu2026omnivoice,
2 title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
3 author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
4 journal={arXiv preprint arXiv:2604.00688},
5 year={2026}
6}