Views
No views yet
| Model | Open-Source | Model Size | test-zh CER (%) ↓ | test-zh Speaker Similarity (%) ↑ | test-en WER (%) ↓ | test-en Speaker Similarity (%) ↑ | test-hard CER (%) ↓ | test-hard Speaker Similarity (%) ↑ |
|---|---|---|---|---|---|---|---|---|
| Human | - | - | 1.26 | 75.5 | 2.14 | 73.4 | - | - |
| Seed-TTS | ❌ | - | 1.12 | 79.6 | 2.25 | 76.2 | 7.59 | 77.6 |
| MiniMax-Speech | ❌ | - | 0.83 | 78.3 | 1.65 | 69.2 | - | - |
| F5-TTS | ✅ | 0.3B | 1.52 | 74.1 | 2.00 | 64.7 | 8.67 | 71.3 |
| Spark TTS | ✅ | 0.5B | 1.2 | 66.0 | 1.98 | 57.3 | - | - |
| CosyVoice2 | ✅ | 0.5B | 1.45 | 75.7 | 2.57 | 65.9 | 6.83 | 72.4 |
| FireRedTTS2 | ✅ | 1.5B | 1.14 | 73.2 | 1.95 | 66.5 | - | - |
| Index-TTS2 | ✅ | 1.5B | 1.03 | 76.5 | 2.23 | 70.6 | 7.12 | 75.5 |
| VibeVoice-1.5B | ✅ | 1.5B | 1.16 | 74.4 | 3.04 | 68.9 | - | - |
| VibeVoice-Realtime | ✅ | 0.5B | - | - | 2.05 | 63.3 | - | - |
| HiggsAudio-v2 | ✅ | 3B | 1.50 | 74.0 | 2.44 | 67.7 | - | - |
| VoxCPM | ✅ | 0.5B | 0.93 | 77.2 | 1.85 | 72.9 | 8.87 | 73.0 |
| GLM-TTS | ✅ | 1.5B | 1.03 | 76.1 | - | - | - | - |
| GLM-TTS RL | ✅ | 1.5B | 0.89 | 76.4 | - | - | - | - |
| Fun-CosyVoice3-0.5B-2512 | ✅ | 0.5B | 1.21 | 78.0 | 2.24 | 71.8 | 6.71 | 75.8 |
| Fun-CosyVoice3-0.5B-2512_RL | ✅ | 0.5B | 0.81 | 77.4 | 1.68 | 69.5 | 5.44 | 75.0 |
1git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
2# If you failed to clone the submodule due to network failures, please run the following command until success
3cd CosyVoice
4git submodule update --init --recursive1conda create -n cosyvoice -y python=3.10
2conda activate cosyvoice
3pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com
4
5# If you encounter sox compatibility issues
6# ubuntu
7sudo apt-get install sox libsox-dev
8# centos
9sudo yum install sox sox-devel1from huggingface_hub import snapshot_download
2snapshot_download('FunAudioLLM/CosyVoice-300M-SFT', local_dir='pretrained_models/CosyVoice-300M-SFT')
3snapshot_download('FunAudioLLM/CosyVoice-ttsfrd', local_dir='pretrained_models/CosyVoice-ttsfrd')ttsfrd resource and install ttsfrd package for better text normalization performance.ttsfrd package, we will use wetext by default.1cd pretrained_models/CosyVoice-ttsfrd/
2unzip resource.zip -d .
3pip install ttsfrd_dependency-0.1-py3-none-any.whl
4pip install ttsfrd-0.4.2-cp310-cp310-linux_x86_64.whl1import sys
2sys.path.append('third_party/Matcha-TTS')
3from cosyvoice.cli.cosyvoice import AutoModel
4import torchaudio
5
6""" CosyVoice Usage, check https://fun-audio-llm.github.io/ for more details
7"""
8cosyvoice = AutoModel(model_dir='pretrained_models/CosyVoice-300M-SFT')
9# sft usage
10print(cosyvoice.list_available_spks())
11# change stream=True for chunk stream inference
12for i, j in enumerate(cosyvoice.inference_sft('CosyVoice is undergoing a comprehensive upgrade, providing more accurate, stable, faster, and better voice generation capabilities.', '中文女', stream=False)):
13 torchaudio.save('sft_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)
14for i, j in enumerate(cosyvoice.inference_sft('你好，我是通义生成式语音大模型，请问有什么可以帮您的吗？', '中文女', stream=False)):
15 torchaudio.save('sft_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)
1@article{du2024cosyvoice,
2 title={Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens},
3 author={Du, Zhihao and Chen, Qian and Zhang, Shiliang and Hu, Kai and Lu, Heng and Yang, Yexin and Hu, Hangrui and Zheng, Siqi and Gu, Yue and Ma, Ziyang and others},
4 journal={arXiv preprint arXiv:2407.05407},
5 year={2024}
6}
7
8@article{du2024cosyvoice,
9 title={Cosyvoice 2: Scalable streaming speech synthesis with large language models},
10 author={Du, Zhihao and Wang, Yuxuan and Chen, Qian and Shi, Xian and Lv, Xiang and Zhao, Tianyu and Gao, Zhifu and Yang, Yexin and Gao, Changfeng and Wang, Hui and others},
11 journal={arXiv preprint arXiv:2412.10117},
12 year={2024}
13}
14
15@article{du2025cosyvoice,
16 title={CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
17 author={Du, Zhihao and Gao, Changfeng and Wang, Yuxuan and Yu, Fan and Zhao, Tianyu and Wang, Hao and Lv, Xiang and Wang, Hui and Shi, Xian and An, Keyu and others},
18 journal={arXiv preprint arXiv:2505.17589},
19 year={2025}
20}
21
22@inproceedings{lyu2025build,
23 title={Build LLM-Based Zero-Shot Streaming TTS System with Cosyvoice},
24 author={Lyu, Xiang and Wang, Yuxuan and Zhao, Tianyu and Wang, Hao and Liu, Huadai and Du, Zhihao},
25 booktitle={ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
26 pages={1--2},
27 year={2025},
28 organization={IEEE}
29}