Westlake-Omni: Open-Source Chinese Emotional Speech Interaction Large Language Model with Unified Discrete Sequence Modeling

🤗 Hugging Face | 📖 Github

Westlake-Omni is an open-source Chinese emotional speech interaction large language model that utilizes discrete representations to achieve unified processing of speech and text modalities. The model supports low-latency generation and high-quality Chinese emotional speech interaction.

🎙️ Utilizes discrete representations to unify the processing of speech and text modalities..

🎭 Trained on a high-quality Chinese emotional speech dataset, enabling native emotional speech interaction in Chinese.

⚡ Low-latency speech interaction, simultaneously generating text and speech responses.

Create a new conda environment and install the required packages:

The current code and the vqgan model weights are provided under the CC-BY-NC-SA-4.0 License. The large language model weights are provided under the Apache 2.0 License. Note that parts of this code are based on Fish speech, released under the CC-BY-NC-SA-4.0 License.

If you have any questions, please raise an issue or contact us at service@xinchenai.com.