Chatterbox-Flash is a block-diffusion zero-shot TTS model that extends the Chatterbox-TTS pipeline with a parallel masked decoder while preserving streaming generation. It unmasks multiple speech tokens in parallel within each block while keeping native block-by-block streaming, delivering autoregressive-class quality at a fraction of the latency.
Streaming & Speed Highlights
Native block-by-block streaming — emits audio as each block is committed, no full-sequence wait.
~9× real-time synthesis at the default config (D = 16, α = 0.5), up to ~13× real-time at D = 32, α = 0.75.
Time-to-first-packet from 103 ms, on par with streaming AR systems.
RTF as low as 0.076 — substantially faster than autoregressive streaming TTS at the same scale.
Early-decoding schedule adaptively ends denoising early, cutting average steps per block by ~20% at negligible quality cost.
Built on a FlashInfer paged KV cache + CUDA-graph inference engine for low per-step overhead.
The released weights cover the four checkpoints needed at inference time:
File
What it is
t3_flash.safetensors
Block-diffusion T3 decoder (Llama-520M + 1 extra [MASK] token).
s3gen.safetensors
Flow-matching S3Gen vocoder.
ve.safetensors
GE2E voice encoder (taken verbatim from ResembleAI/chatterbox).
tokenizer.json
English BPE tokenizer (taken verbatim from ResembleAI/chatterbox).
Quick start
pip install chatterbox-flash
python
1import torchaudio as ta
2from chatterbox_flash import ChatterboxFlashTTS
3tts = ChatterboxFlashTTS.from_pretrained("ResembleAI/chatterbox-flash", device="cuda")4wav = tts.generate(5"Hello, world.",6 audio_prompt_path="reference.wav",7)8ta.save("out.wav", wav.unsqueeze(0).cpu(), tts.sr)
Inference defaults (paper configuration)
Block size D = 16
Maximum K = 10 denoising steps per block
Sampling temperature 0.2
shift outlier schedule with tau = 0.5
CFG with w = 1.0, pmi_cfg combination
FlashInfer paged KV cache + CUDA graph capture
Streaming Efficiency
Latency and throughput at concurrency 1, measured over 50 utterances. TTFP is the wall-clock time from request to the first emitted audio packet; RTF (real-time factor) is generation time divided by synthesized audio duration — lower is faster, and RTF < 1 means faster than real time.
Config (25 Hz, 0.5B)
TTFP (ms) ↓
RTF ↓
D = 16, α = 0.5 (default)
118
0.107
D = 16, α = 0.75
106
0.091
D = 24, α = 0.5
119
0.100
D = 24, α = 0.75
105
0.084
D = 32, α = 0.5
115
0.090
D = 32, α = 0.75
103
0.076
Even on a single concurrent request, Chatterbox-Flash sustains roughly 9× real-time synthesis at the default setting and ~13× real-time at D = 32, α = 0.75, while keeping time-to-first-packet low enough for interactive streaming.
Apple Silicon (MLX)
Chatterbox-Flash also runs locally on Apple Silicon via MLX. The numbers below were measured on a Mac M4 at the default configuration; both stay comfortably under real time (RTF < 1), and 4-bit quantization gives a further speedup.
Backend
RTF ↓
MLX
0.778
MLX (4-bit quantized)
0.665
Quality (Seed-TTS test-en)
Zero-shot TTS quality on the Seed-TTS English benchmark, under the canonical configuration (D = 16). SIM-o is speaker similarity to the reference (higher is better), WER is word error rate from ASR transcription (lower is better), and UTMOS is a predicted naturalness score (higher is better). Results are shown for our two main decoding settings against the Chatterbox backbone and ground-truth audio for reference.
System
SIM-o ↑
WER ↓
UTMOS ↑
Ground-truth
0.734
2.14
3.52
Chatterbox (AR backbone)
0.685
2.20
4.10
Chatterbox-Flash (α = 0)
0.704
1.96
4.09
Chatterbox-Flash (α = 0.5, early decoding)
0.704
2.04
4.08
Converting the autoregressive backbone into a block-diffusion decoder improves both speaker similarity (0.685 → 0.704) and intelligibility (2.20 → 1.96 WER) while keeping naturalness essentially unchanged — all while unlocking parallel, streaming-friendly decoding.
