WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS
WordVoice is a novel speech generation framework that transforms traditional implicit end-to-end TTS generation into an explicit, highly controllable paradigm. Built on top of the CosyVoice3 framework, it enables precise and decoupled word-level control over five acoustic dimensions.
Employs a bound-token (<b>) mechanism within the autoregressive (AR) language model. Before generating the speech tokens for a specific word, the model explicitly predicts its acoustic attributes, realizing an intelligent process of "planning prosody first, then generating sound."
🛠️ Quick Start
Installation
We recommend using Conda to manage your Python environment.
Run the following script to automatically download the pre-trained weights and dependencies (such as CosyVoice3, MMS-FA, etc.):
bash download_models.sh
Inference
You can run the out-of-the-box inference script to experience both the Free Mode and Control Mode of WordVoice:
python wordvoice_infer.py
For custom prompts and detailed control parameters, refer to wordvoice_infer.py and the GitHub repository.
📝 Citation
If you find this work or the models useful, please cite:
bibtex
1@misc{nie2026wordvoice,
2 title={WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS},
3 author={Sihang Nie and Jinxin Ji and Xiaofen Xing and Deyi Tuo and Chengbin Jin and Jialong Mai and Xiangmin Xu},
4 year={2026},
5 eprint={2607.06461},
6 archivePrefix={arXiv},
7 primaryClass={eess.AS},
8 url={https://arxiv.org/abs/2607.06461},
9}
