You can deploy this model and start earning money today!

Audio8/
audio8-TTS-0.1B-ONNX-INT8

Name: audio8-TTS-0.1B-ONNX-INT8
Availability: InStock

Views

No views yet

Audio8 TTS Preview 0.1B ONNX INT8

A compact multilingual TTS with zero-shot voice cloning, packaged for low-resource CPU inference.

GitHub

Demo

ONNX Runtime

Audio8 TTS Preview is a compact multilingual text-to-speech model with zero-shot voice cloning. This repository provides its CPU-oriented ONNX deployment: INT8 per-token DualAR models, an FP16 neural audio codec, the tokenizer, the packaged reference voice, and the optional FP16 encoder used to register reference voices.

Model files only. Inference, streaming service, and voice-registration code live in the dedicated 0.1B INT8 ONNX Runtime. The older onnx_runtime/ directory targets the incompatible 0.6B INT4 graphs and must not be used with this model.

Acknowledgments

The per-token ONNX export design (stacked recurrent interface, valid-prefix attention) and the INT8 quantization strategy in this repository were adapted from the community conversion Masterx/Audio8-TTS-Preview-0.1B-ONNX-INT8. The graphs and tooling are independently produced by the Audio8 team from the official checkpoint and do not reuse that repository's artifacts. Thanks to Masterx and the community for exploring ONNX Runtime conversion of the 0.1B model.

Why this ONNX release

	Deployment characteristic
CPU native	ONNX Runtime `CPUExecutionProvider`; no CUDA requirement
Small runtime	No PyTorch, Transformers, or Hugging Face Hub dependency after download
Low memory	About 0.6 GB after loading the online sessions in our test configuration
Voice cloning	Bundled FP16 codec encoder for reusable local voice profiles
Local service	CLI, web UI, HTTP API, streaming PCM, and OpenAI-compatible endpoint

Precision and footprint

Component	Precision
Slow/Fast AR	INT8 (dynamic activation quantization, `MatMulInteger`)
Codec encoder and decoder	FP16
Waveform output	FP32, 44.1 kHz mono

The slow AR runs as a per-token recurrent graph (attention + Mamba states passed explicitly) so the CPU workload stays tiny: about 19 ms per slow-AR token and 8 ms per fast-AR frame on our test host (8 threads). A one-time prompt prefill costs about 19 ms per prompt token. Normal synthesis loads only the Slow AR, Fast AR, and codec decoder sessions; the online model files occupy about 0.6 GB. The complete repository, including the voice-registration encoder, is about 1.0 GB. Actual memory use varies by platform and allocator behavior.

Supported Languages

Cantonese · Chinese · Dutch · English
French · German · Italian · Japanese
Korean · Polish · Spanish

Preview status: Language coverage is intentionally limited in this release. For the best results, use one of the 11 recommended languages above. Broader multilingual coverage and Chinese dialect support are planned for future releases.

Model Details

Audio8 TTS uses a DualAR architecture inspired by Fish Audio S2 Pro. The 0.1B model uses a Falcon-H1 hybrid slow AR (attention + Mamba) that predicts one semantic token per audio frame; the fast AR predicts the frame's codec codebooks, conditioned on the slow hidden state and preceding codebooks.

Component	Configuration
Main model	about 100M parameters, excluding the codec
Slow AR	24 layers, width 512, Falcon-H1 hybrid (8 attention heads, 2 KV heads, Mamba d_state 64)
Fast AR	4 layers, width 512
Acoustic tokens	10 codebooks, 4,096 entries per codebook
Codec	44.1 kHz, 2,048 samples per model frame (~21.5 frames/s)
Context	Up to 2,048 packed text/audio positions
Execution provider	ONNX Runtime CPU

Quick Start

Python 3.10 or newer is required. The runtime uses ONNX Runtime's CPUExecutionProvider; platform-specific ONNX Runtime wheels may be needed on devices such as Jetson. The current release is tested on macOS arm64 and Linux x86_64.

1. Download the code and model

bash

git clone https://github.com/Audio8-AI/Audio8_TTS.git
cd Audio8_TTS/onnx_runtime_0_1b_int8

python3 -m pip install -U "huggingface_hub[cli]"
hf download Audio8/audio8-TTS-0.1B-ONNX-INT8 --local-dir model
bash setup.sh
"$PWD/.venv/bin/python" scripts/register_default_voice.py   # packaged reference -> "default" voice

The model files are stored at this Hugging Face repository's root. Downloading with --local-dir model creates the exact layout expected by the runtime:

text

model/
├── slow_ar_int8.onnx          # per-token slow AR (prefill + decode)
├── fast_ar_int8.onnx          # per-token fast AR
├── codec_decoder_fp16.onnx(.data)
├── runtime_manifest.json
├── tokenizer/tokenizer.json
├── reference_codes.npy        # packaged fixed reference
└── registration/
    ├── codec_encoder_fp16.onnx(.data)
    └── registration_manifest.json

2. Generate speech

bash

bash run_infer.sh \
  --voice default \
  --text "这是一个零样本语音合成的中文测试，声音克隆效果怎么样？" \
  --max-new-tokens 256 \
  --output outputs/example.wav

The command writes outputs/example.wav and [10, T] codec codes to outputs/example.npy.

3. Register a reference voice

Start the local service and open http://127.0.0.1:8024. The page provides text generation, voice selection, WAV playback/download, reference-voice registration, memory status, and runtime reload. The Swagger API documentation is available at http://127.0.0.1:8024/docs. Upload a 0.5-30 second reference recording, its exact transcript, and a voice name to register a new voice.

bash start_server.sh

The same operation is available through HTTP:

bash

curl http://127.0.0.1:8024/api/voices/register \
  -F 'audio=@/absolute/path/reference.wav' \
  -F 'text=The exact transcript of the reference recording.' \
  -F 'name=speaker_a' \
  -F 'overwrite=false'

HTTP API

bash

curl http://127.0.0.1:8024/api/tts \
  -H 'Content-Type: application/json' \
  -d '{"text":"Welcome to Audio8 TTS.","voice_name":"speaker_a","max_new_tokens":256}' \
  -o outputs/api.wav

OpenAI-compatible API

bash

curl http://127.0.0.1:8024/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"arktts","input":"Welcome to Audio8 TTS.","voice":"speaker_a","response_format":"wav"}' \
  -o outputs/openai.wav

See the complete 0.1B ONNX Runtime guide for streaming output, configuration, memory management, and service controls.

Evaluation

The source Audio8 TTS Preview 0.1B checkpoint is a compact multilingual model; see the base model card for details.

INT8 quantization can change sampled token sequences, so quality should be evaluated for each target language, voice, and deployment setting rather than assuming bit-for-bit equivalence with the source checkpoint. This repository contains the 0.1B INT8 graphs; use the dedicated runtime linked above when running them.

Limitations and Responsible Use

This is a Preview checkpoint with limited multilingual and dialect coverage.
Very long, noisy, or incorrectly transcribed references can reduce stability and speaker similarity.
Generated speech can be misused for impersonation or misinformation. Obtain consent before cloning a voice and clearly disclose synthetic audio where appropriate.
Evaluate the model for accuracy, safety, and legal compliance before deployment.

License and Acknowledgements

The code and model weights are released under the Apache License 2.0. See the upstream NOTICE for attribution details.

We thank the Fish Audio team for publishing the DualAR architecture used in Fish Audio S2 Pro.