Zero-shot TTS: Input a 5-second vocal sample and experience instant text-to-speech conversion.
Few-shot TTS: Fine-tune the model with just 1 minute of training data for improved voice similarity and realism.
Cross-lingual Support: Inference in languages different from the training dataset, currently supporting English, Japanese, Korean, Cantonese and Chinese.
WebUI Tools: Integrated tools include voice accompaniment separation, automatic training set segmentation, Chinese ASR, and text labeling, assisting beginners in creating training datasets and MoYoYo.tts models.
RTF(inference speed) of MoYoYo.tts v2 ProPlus:
0.028 tested in 4060Ti, 0.014 tested in 4090 (1400words~=4min, inference time is 3.36s), 0.526 in M4 CPU. You can test our huggingface demo (half H200) to experience high-speed inference .
Note: The models trained with GPUs on Macs result in significantly lower quality compared to those trained on other devices, so we are temporarily using CPUs instead.
Install the program by running the following commands:
Due to rapid development in the codebase and a slower Docker image release cycle, please:
Check Docker Hub for the latest available image tags
Choose an appropriate image tag for your environment
Lite means the Docker image does not include ASR models and UVR5 models. You can manually download the UVR5 models, while the program will automatically download the ASR models as needed
The appropriate architecture image (amd64/arm64) will be automatically pulled during Docker Compose
Docker Compose will mount all files in the current directory. Please switch to the project root directory and pull the latest code before using the Docker image
Optionally, build the image locally using the provided Dockerfile for the most up-to-date changes
Environment Variables
is_half: Controls whether half-precision (fp16) is enabled. Set to true if your GPU supports it to reduce memory usage.
Shared Memory Configuration
On Windows (Docker Desktop), the default shared memory size is small and may cause unexpected behavior. Increase shm_size (e.g., to 16g) in your Docker Compose file based on your available system memory.
Choosing a Service
The docker-compose.yaml defines two services:
GPT-SoVITS-CU126 & GPT-SoVITS-CU128: Full version with all features.
GPT-SoVITS-CU126-Lite & GPT-SoVITS-CU128-Lite: Lightweight version with reduced dependencies and functionality.
To run a specific service with Docker Compose, use:
docker compose run --service-ports <GPT-SoVITS-CU126-Lite|GPT-SoVITS-CU128-Lite|GPT-SoVITS-CU126|GPT-SoVITS-CU128>
Building the Docker Image Locally
If you want to build the image yourself, use:
bash docker_build.sh --cuda <12.6|12.8> [--lite]
Accessing the Running Container (Bash Shell)
Once the container is running in the background, you can access it using:
For UVR5 (Vocals/Accompaniment Separation & Reverberation Removal, additionally), download models from UVR5 Weights and place them in tools/uvr5/uvr5_weights.
If you want to use bs_roformer or mel_band_roformer models for UVR5, you can manually download the model and corresponding configuration file, and put them in tools/uvr5/uvr5_weights. Rename the model file and configuration file, ensure that the model and configuration files have the same and corresponding names except for the suffix. In addition, the model and configuration file names must include roformer in order to be recognized as models of the roformer class.
The suggestion is to directly specify the model type in the model name and configuration file name, such as mel_mand_roformer, bs_roformer. If not specified, the features will be compared from the configuration file to determine which type of model it is. For example, the model bs_roformer_ep_368_sdr_12.9628.ckpt and its corresponding configuration file bs_roformer_ep_368_sdr_12.9628.yaml are a pair, kim_mel_band_roformer.ckpt and kim_mel_band_roformer.yaml are also a pair.
For English or Japanese ASR (additionally), download models from Faster Whisper Large V3 and place them in tools/asr/models. Also, other models may have the similar effect with smaller disk footprint.
Dataset Format
The TTS annotation .list file format:
vocal_path|speaker_name|language|text
Language dictionary:
'zh': Chinese
'ja': Japanese
'en': English
'ko': Korean
'yue': Cantonese
Example:
D:\GPT-SoVITS\xxx/xxx.wav|xxx|en|I like playing Genshin.
Finetune and inference
Open WebUI
Integrated Package Users
Double-click go-webui.bator use go-webui.ps1
if you want to switch to V1,then double-clickgo-webui-v1.bat or use go-webui-v1.ps1
Others
python webui.py <language(optional)>
if you want to switch to V1,then
python webui.py v1 <language(optional)>
Or maunally switch version in WebUI
Finetune
Path Auto-filling is now supported
Fill in the audio path
Slice the audio into small chunks
Denoise(optinal)
ASR
Proofreading ASR transcriptions
Go to the next Tab, then finetune the model
Open Inference WebUI
Integrated Package Users
Double-click go-webui-v2.bat or use go-webui-v2.ps1 ,then open the inference webui at 1-GPT-SoVITS-TTS/1C-inference
The timbre similarity is higher, requiring less training data to approximate the target speaker (the timbre similarity is significantly improved using the base model directly without fine-tuning).
GPT model is more stable, with fewer repetitions and omissions, and it is easier to generate speech with richer emotional expression.
pip install -r requirements.txt to update some packages
Clone the latest codes from github.
Download v3 pretrained models (s1v3.ckpt, s2Gv3.pth and models--nvidia--bigvgan_v2_24khz_100band_256x folder) from huggingface and put them into GPT_SoVITS/pretrained_models.
additional: for Audio Super Resolution model, you can read how to download
V4 Release Notes
New Features:
Version 4 fixes the issue of metallic artifacts in Version 3 caused by non-integer multiple upsampling, and natively outputs 48k audio to prevent muffled sound (whereas Version 3 only natively outputs 24k audio). The author considers Version 4 a direct replacement for Version 3, though further testing is still needed.
more details
Use v4 from v1/v2/v3 environment:
pip install -r requirements.txt to update some packages
Clone the latest codes from github.
Download v4 pretrained models (gsv-v4-pretrained/s2v4.pth, and gsv-v4-pretrained/vocoder.pth) from huggingface and put them into GPT_SoVITS/pretrained_models.
V2Pro Release Notes
New Features:
Slightly higher VRAM usage than v2, surpassing v4's performance, with v2's hardware cost and speed.
more details
2.v1/v2 and the v2Pro series share the same characteristics, while v3/v4 have similar features. For training sets with average audio quality, v1/v2/v2Pro can deliver decent results, but v3/v4 cannot. Additionally, the synthesized tone and timebre of v3/v4 lean more toward the reference audio rather than the overall training set.
Use v2Pro from v1/v2/v3/v4 environment:
pip install -r requirements.txt to update some packages
Clone the latest codes from github.
Download v2Pro pretrained models (v2Pro/s2Dv2Pro.pth, v2Pro/s2Gv2Pro.pth, v2Pro/s2Dv2ProPlus.pth, v2Pro/s2Gv2ProPlus.pth, and sv/pretrained_eres2netv2w24s4ep4.ckpt) from huggingface and put them into GPT_SoVITS/pretrained_models.
