Original model: neuphonic/neutts-2e by Neuphonic.

Community ONNX export (FP32 + dynamic-int8) of NeuTTS-2E, Neuphonic's on-device emotional English TTS model (~236M params, Qwen3-architecture backbone + NeuCodec).

A 4-bit MatMulNBits variant was also built and tested but is not shipped: it decoded at the same speed as int8 (45.1 vs 45.3 tok/s) while being 1.7× larger and less accurate (argmax agreement 14/21 on teacher-forced steps), with no configuration (fp32/int8 lm_head, block size 32) beating int8 on any axis.

Measured with bench_onnx.py on an idle AMD Ryzen Threadripper 7970X (32C/64T), onnxruntime 1.27.0 (CPU EP), prompt 454 tokens, 250 decode steps. Decode timing includes top-k/top-p/min-p sampling over the 217k vocab; it excludes prefill (shown separately) and NeuCodec decode (~0.2 s per 5 s of audio, ~4%). NeuCodec produces 50 frames/s of audio, so ×RT = tok/s ÷ 50 (higher is better).

Set intra_op_num_threads to 3–4. Decode throughput peaks there and collapses at higher counts (the ONNX Runtime default — all cores — is 2–3× slower than the peak for this small-hidden-size model). onnx_infer.py defaults to 4 threads (--threads).

Exported with optimum-cli export onnx --task text-generation-with-past (optimum 2.1.0 / optimum-onnx 0.1.0, fp32). Export validated against the torch model: fp32 prefill logits max abs diff 3.8e-05 with identical argmax, and 21 teacher-forced decode steps with 21/21 argmax agreement and 10/10 top-10 overlap. Note the neutts library runs the original backbone in bfloat16, which itself deviates from the fp32 reference by up to 0.136 in logit space — the FP32 ONNX model is numerically closer to the fp32 weights than the official pipeline. The int8 model is a dynamic-range quantization (onnxruntime.quantization.quantize_dynamic, QInt8 weights).

Because ONNX cannot tie weights, the shared embedding/LM-head matrix of the original model is materialized twice in the FP32 graph — that is why the FP32 file is larger than the original safetensors. One config change was made for export tooling compatibility: the transformers-5 rope_parameters block was flattened back to the legacy rope_theta: 10000.0 key.

The neutts library does not support ONNX backbones directly (only ONNX codec decoders), so this repo ships onnx_infer.py, which reproduces the library's prompt format exactly (verified token-for-token against neutts' _apply_chat_template, including the emotion-token path and quote normalization).

onnx_infer.py samples with temperature 1.0, top-k 50, top-p 0.95, min-p 0.05. The official torch backend uses pure top-k 50; the official GGUF path additionally inherits llama.cpp's default top-p 0.95 / min-p 0.05, and in A/B testing that chain scored slightly better (mean +0.21 PESQ over 4 matched pairs), so it is the default here — pass --top-p 1.0 --min-p 0 for exact torch-backend behavior. min_new_tokens=50 forces ≥1 s of audio even for very short inputs (matching the library), and the 2048-token context bounds a single generation to roughly 30 s of audio. As with the original model, output is stochastic: occasional bad draws (a slurred word, a trailing artifact) happen at any precision — re-run with a different --seed if a take sounds off.

NeuCodec weights are fetched from the Hub on first run. Pre-encoded speaker references are loaded from the installed neutts package. Use --model model_int8.onnx for the faster int8 variant and --codec neuphonic/neucodec-onnx-decoder to run the codec decoder under ONNX Runtime as well (torch is still required to load the .pt speaker references and by the neucodec/neutts packages).

Generated with the FP32 model.onnx in this repo (default sampler above, seed 1234 — except paul/sophie neutral, re-rolled with seed 4321), decoded with the torch NeuCodec decoder, Perth-watermarked. All four voices are included, each with a neutral sample and one emotional sample, plus two samples from model_int8.onnx for quality comparison. Every sample below was verified to transcribe exactly back to its input text with Whisper (base.en).

Distributed under the NeuTTS Open License v1.0 (free research use; commercial use limited to entities under $5M annual revenue — see LICENSE). Modification notice per the license: this repository contains an ONNX export (and an int8-quantized variant) of the original safetensors weights; conversion changes are described above.