Fish Audio S2 is an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and instruction-following control via natural-language descriptions. The system utilizes a multi-stage training recipe and a staged data pipeline covering video and speech captioning. S2 Pro specifically uses a Dual-Autoregressive (Dual-AR) architecture:
Slow AR (4B): Predicts the primary semantic codebook along the time axis.
Fast AR (400M): Generates the remaining residual codebooks to reconstruct fine-grained acoustic detail.
What is this?
This is a weight-only FP8 quantization of Fish Audio S2 Pro — a state-of-the-art open-source TTS model with fine-grained inline prosody and emotion control across 80+ languages. The quantization cuts the on-disk size roughly in half and reduces VRAM usage from ~24 GB to ~12 GB, with no perceptible quality loss in practice.
Original (s2-pro)
This (s2-pro-fp8)
Weight dtype
bfloat16
float8_e4m3fn
Activation dtype
bfloat16
bfloat16
Scale
—
per-row float32
File size
~12 GB
~6.2 GB
VRAM (inference)
~24 GB
~12 GB
Extra dependencies
none
none
Quantization Details
What is quantized: All nn.Linear weight matrices in both the Slow AR (4B) and Fast AR (400M) backbones — 201 layers in total. Non-linear weights (embeddings, layer norms, codec) remain in bfloat16.
Method: Per-row symmetric FP8
Each output row of every weight matrix has its own float32 scale factor:
Per-row scaling captures the per-channel magnitude variation in transformer weight matrices much better than a single per-tensor scale, significantly reducing quantization error at minimal overhead.
No external quantization library required. Dequantization is implemented in pure PyTorch inside a custom FP8Linear module — no torchao, bitsandbytes, or AutoGPTQ needed. The model loads and runs on any machine with PyTorch 2.1+.
All other tensors — bfloat16 (embeddings, norms, codec, etc.)
Hardware Requirements
GPU: NVIDIA GPU with CUDA support
VRAM: ~12 GB
Native FP8 tensor cores: Ada Lovelace or Blackwell (RTX 4090, RTX 5090, H100, etc.) — recommended for full speed
Older GPUs (Ampere and below): Will load and run correctly. Dequantization to bfloat16 happens on all hardware, so you still get the ~12 GB VRAM footprint benefit even without native FP8 cores.
Usage — ComfyUI (Recommended)
The easiest way to use this model is with ComfyUI-FishAudioS2, which has native support for this FP8 model with zero extra setup.
Installation
Install the ComfyUI node via ComfyUI Manager (search FishAudioS2) or manually:
precision: auto or bfloat16 — matches the activation dtype
attention: auto or sage_attention for best performance
keep_model_loaded: True if running multiple generations back-to-back
Works with all three nodes: Fish S2 TTS, Fish S2 Voice Clone TTS, Fish S2 Multi-Speaker TTS.
About Fish Audio S2 Pro
Fish Audio S2 Pro is a leading text-to-speech model with fine-grained inline control of prosody and emotion. Trained on over 10M+ hours of audio data across 80+ languages, it combines reinforcement learning alignment with a Dual-Autoregressive (Dual-AR) architecture.
Architecture
S2 Pro builds on a decoder-only transformer combined with an RVQ-based audio codec (10 codebooks, ~21 Hz frame rate):
Slow AR (4B parameters): Operates along the time axis and predicts the primary semantic codebook.
Fast AR (400M parameters): Generates the remaining 9 residual codebooks at each time step, reconstructing fine-grained acoustic detail.
This asymmetric design keeps inference efficient while preserving audio fidelity. The Dual-AR architecture is structurally isomorphic to standard autoregressive LLMs, inheriting LLM-native serving optimizations — continuous batching, paged KV cache, CUDA graph replay, RadixAttention-based prefix caching.
Fine-Grained Inline Control
Embed natural-language instructions directly in the text using [tag] syntax. S2 Pro accepts free-form descriptions — not a fixed tag vocabulary:
Free-form examples:[whisper in small voice] · [super happy and excited] · [speaking slowly and clearly] · [sarcastic tone]
15,000+ unique tags supported.
Supported Languages
Tier 1 (Best Quality): Japanese (ja), English (en), Chinese (zh)
Tier 2: Korean (ko), Spanish (es), Portuguese (pt), Arabic (ar), Russian (ru), French (fr), German (de)
80+ total: sv, it, tr, no, nl, cy, eu, ca, da, gl, ta, hu, fi, pl, et, hi, la, ur, th, vi, jw, bn, yo, sl, cs, sw, nn, he, ms, uk, id, kk, bg, lv, my, tl, sk, ne, fa, af, el, bo, hr, ro, sn, mi, yi, am, be, km, is, az, sd, br, sq, ps, mn, ht, ml, sr, sa, te, ka, bs, pa, lt, kn, si, hy, mr, as, gu, fo
Production Streaming Performance (original model, H200)
Real-Time Factor (RTF): 0.195
Time-to-first-audio: ~100 ms
Throughput: 3,000+ acoustic tokens/s while maintaining RTF below 0.5
Citation
bibtex
1@misc{liao2026fishaudios2technical,
2 title={Fish Audio S2 Technical Report},
3 author={Shijia Liao and Yuxuan Wang and Songting Liu and Yifan Cheng and Ruoyi Zhang and Tianyu Li and Shidong Li and Yisheng Zheng and Xingwei Liu and Qingzheng Wang and Zhizhuo Zhou and Jiahua Liu and Xin Chen and Dawei Han},
4 year={2026},
5 eprint={2603.08823},
6 archivePrefix={arXiv},
7 primaryClass={cs.SD},
8 url={https://arxiv.org/abs/2603.08823},
9}
