You can deploy this model and start earning money today!

iomaar/
F5-TTS

Name: F5-TTS
Availability: InStock

Views

No views yet

F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

arXiv

demo

F5-TTS: Diffusion Transformer with ConvNeXt V2, faster trained and inference.

E2 TTS: Flat-UNet Transformer, closest reproduction from paper.

Sway Sampling: Inference-time flow step sampling strategy, greatly improves performance

Thanks to all the contributors !

News

2025/03/12: 🔥 F5-TTS v1 base model with better training and inference performance. Few demo.
2024/10/08: F5-TTS & E2 TTS base models on 🤗 Hugging Face, 🤖 Model Scope, 🟣 Wisemodel.

Installation

Create a separate environment if needed

bash

# Create a python 3.10 conda env (you could also use virtualenv)
conda create -n f5-tts python=3.10
conda activate f5-tts

Install PyTorch with matched device

NVIDIA GPU

bash
1# Install pytorch with your CUDA version, e.g. 2pip install torch==2.4.0+cu124 torchaudio==2.4.0+cu124 --extra-index-url https://download.pytorch.org/whl/cu124

AMD GPU

bash
1# Install pytorch with your ROCm version (Linux only), e.g. 2pip install torch==2.5.1+rocm6.2 torchaudio==2.5.1+rocm6.2 --extra-index-url https://download.pytorch.org/whl/rocm6.2

Intel GPU

bash
1# Install pytorch with your XPU version, e.g. 2# Intel® Deep Learning Essentials or Intel® oneAPI Base Toolkit must be installed 3pip install torch torchaudio --index-url https://download.pytorch.org/whl/test/xpu 4 5# Intel GPU support is also available through IPEX (Intel® Extension for PyTorch) 6# IPEX does not require the Intel® Deep Learning Essentials or Intel® oneAPI Base Toolkit 7# See: https://pytorch-extension.intel.com/installation?request=platform

Apple Silicon

bash
1# Install the stable pytorch, e.g. 2pip install torch torchaudio

Then you can choose one from below:

1. As a pip package (if just for inference)

pip install f5-tts

2. Local editable (if also do training, finetuning)

bash
1git clone https://github.com/SWivid/F5-TTS.git 2cd F5-TTS 3# git submodule update --init --recursive # (optional, if need > bigvgan) 4pip install -e .

Docker usage also available

bash

# Build from Dockerfile
docker build -t f5tts:v1 .

# Run from GitHub Container Registry
docker container run --rm -it --gpus=all --mount 'type=volume,source=f5-tts,target=/root/.cache/huggingface/hub/' -p 7860:7860 ghcr.io/swivid/f5-tts:main

# Quickstart if you want to just run the web interface (not CLI)
docker container run --rm -it --gpus=all --mount 'type=volume,source=f5-tts,target=/root/.cache/huggingface/hub/' -p 7860:7860 ghcr.io/swivid/f5-tts:main f5-tts_infer-gradio --host 0.0.0.0

Runtime

Deployment solution with Triton and TensorRT-LLM.

Benchmark Results

Decoding on a single L20 GPU, using 26 different prompt_audio & target_text pairs.

Model	Concurrency	Avg Latency	RTF	Mode
F5-TTS Base (Vocos)	2	253 ms	0.0394	Client-Server
F5-TTS Base (Vocos)	1 (Batch_size)	-	0.0402	Offline TRT-LLM
F5-TTS Base (Vocos)	1 (Batch_size)	-	0.1467	Offline Pytorch

See detailed instructions for more information.

Inference

In order to achieve desired performance, take a moment to read detailed guidance.
By properly searching the keywords of problem encountered, issues are very helpful.

1. Gradio App

Currently supported features:

Basic TTS with Chunk Inference
Multi-Style / Multi-Speaker Generation
Voice Chat powered by Qwen2.5-3B-Instruct
Custom inference with more language support

bash

# Launch a Gradio app (web interface)
f5-tts_infer-gradio

# Specify the port/host
f5-tts_infer-gradio --port 7860 --host 0.0.0.0

# Launch a share link
f5-tts_infer-gradio --share

NVIDIA device docker compose file example

yaml

services:
  f5-tts:
    image: ghcr.io/swivid/f5-tts:main
    ports:
      - "7860:7860"
    environment:
      GRADIO_SERVER_PORT: 7860
    entrypoint: ["f5-tts_infer-gradio", "--port", "7860", "--host", "0.0.0.0"]
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  f5-tts:
    driver: local

2. CLI Inference

bash

# Run with flags
# Leave --ref_text "" will have ASR model transcribe (extra GPU memory usage)
f5-tts_infer-cli --model F5TTS_v1_Base \
--ref_audio "provide_prompt_wav_path_here.wav" \
--ref_text "The content, subtitle or transcription of reference audio." \
--gen_text "Some text you want TTS model generate for you."

# Run with default setting. src/f5_tts/infer/examples/basic/basic.toml
f5-tts_infer-cli
# Or with your own .toml file
f5-tts_infer-cli -c custom.toml

# Multi voice. See src/f5_tts/infer/README.md
f5-tts_infer-cli -c src/f5_tts/infer/examples/multi/story.toml

Training

1. With Hugging Face Accelerate

Refer to training & finetuning guidance for best practice.

2. With Gradio App

bash

# Quick start with Gradio web interface
f5-tts_finetune-gradio

Read training & finetuning guidance for more instructions.

Evaluation

Development

Use pre-commit to ensure code quality (will run linters and formatters automatically):

bash

pip install pre-commit
pre-commit install

When making a pull request, before each commit, run:

pre-commit run --all-files

Note: Some model components have linting exceptions for E722 to accommodate tensor notation.

Acknowledgements

E2-TTS brilliant work, simple and effective
Emilia, WenetSpeech4TTS, LibriTTS, LJSpeech valuable datasets
lucidrains initial CFM structure with also bfs18 for discussion
SD3 & Hugging Face diffusers DiT and MMDiT code structure
torchdiffeq as ODE solver, Vocos and BigVGAN as vocoder
FunASR, faster-whisper, UniSpeech, SpeechMOS for evaluation tools
ctc-forced-aligner for speech edit test
mrfakename huggingface space demo ~
f5-tts-mlx Implementation with MLX framework by Lucas Newman
F5-TTS-ONNX ONNX Runtime version by DakeQQ
Yuekai Zhang Triton and TensorRT-LLM support ~

Citation

If our work and codebase is useful for you, please cite as:

@article{chen-etal-2024-f5tts,
      title={F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching}, 
      author={Yushen Chen and Zhikang Niu and Ziyang Ma and Keqi Deng and Chunhui Wang and Jian Zhao and Kai Yu and Xie Chen},
      journal={arXiv preprint arXiv:2410.06885},
      year={2024},
}

License

Our code is released under MIT License. The pre-trained models are licensed under the CC-BY-NC license due to the training data Emilia, which is an in-the-wild dataset. Sorry for any inconvenience this may cause.

iomaar/F5-TTS

F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

Thanks to all the contributors !

News

Installation

Create a separate environment if needed

Install PyTorch with matched device

Then you can choose one from below:

1. As a pip package (if just for inference)

2. Local editable (if also do training, finetuning)

Docker usage also available

Runtime

Benchmark Results

Inference

1. Gradio App

2. CLI Inference

Training

1. With Hugging Face Accelerate

2. With Gradio App

Evaluation

Development

Acknowledgements

Citation

License

iomaar/
F5-TTS