CoreML conversion of Supertone/supertonic-3,
a 99M-parameter multilingual TTS model. All 4 components run on the
Apple Neural Engine (1.8–3.7× faster than CPU on M-series chips).
Component
Size
Role
fp16/duration_predictor.mlpackage
15 MB
text -> frame count
fp16/text_encoder.mlpackage
71 MB
text -> conditioning latent
fp16/vector_estimator.mlpackage
135 MB
flow-matching denoiser (8 steps)
fp16/vocoder.mlpackage
51 MB
latent -> 44.1 kHz waveform
Total
272 MB
(originals: ~400 MB ONNX)
Quickstart
bash
1pip install coremltools soundfile numpy supertonic
2git clone https://huggingface.co/Reza2kn/supertonic-3-coreml
3cd supertonic-3-coreml
45# Short prompt6python inference.py --text "Hello, world." --voice F1 --lang en --out hello.wav
78# Long prompt — use --auto-pad for full content rendering9python inference.py \10 --text "A gentle breeze moved through the open window while everyone listened to the story. The narrator paused, took a slow breath, and continued in a softer tone. Outside, the city carried on, unaware of the quiet moment unfolding inside."\11 --voice F5 --lang en --auto-pad --out long.wav
10 voice styles ship in voice_styles/: F1–F5 (female), M1–M5 (male).
31 languages supported via unicode_indexer.json.
The auto-pad trick (why --auto-pad matters)
The supertonic-3 model has a soft cap on how much speech it renders per
utterance. For long inputs (more than ~13 s of natural speech) the model
truncates the prompt and emits a low-amplitude filler tone for the rest
of the budget. The CoreML conversion's static bucket (T=L=320) extends
this cap by ~3 s due to the way the bucket's padded positions leak into
the real positions through ConvNeXt's dilated convolutions — that's
why CoreML inference sounds more natural than the original ONNX
library (proper word separation, intonation), but it still cuts off
mid-sentence on long prompts.
--auto-pad is a two-pass workaround:
Pass 1 synthesizes the prompt alone at full bucket length to find
where the model's content naturally stops (t_orig).
Pass 2 appends a long filler sentence
(" And with that, the gentle silence wrapped itself around the room.")
that gives the model extra frames to fully render the original
prompt, then renders the filler sentence, then drops into the filler
tone.
The longest clean-silence gap after t_orig is the boundary between
the original prompt and the appended filler. The pipeline trims
there and tail-pads with 0.5 s of true silence.
Cost: ~2× synthesis time. Worth it for any prompt over ~5 s.
ANE engagement
All 4 components compile to ANE-resident programs when loaded with
compute_units=ALL (default). Measured speedups on M2 Pro vs CPU:
Static bucket: T=320 (text length), L=320 (latent length). Inputs are
zero-padded on the right and masked. Bucket = 22.3 s of audio.
duration_predictor, text_encoder, vocoder are hand-reimplemented
in PyTorch from the ONNX initializers, then traced to CoreML.
Per-component float parity vs ONNX: max-abs 2e-5 (dp), cos 0.998
(text_encoder), cos 0.9998 (vocoder).
vector_estimator (the heavy diffusion model) goes through
onnxsim.simplify(T=L=320) -> onnx2torch.convert -> torch.jit.trace
-> coremltools. Cos 0.998 vs ONNX per diffusion step.
The diffusion sampler stays host-side (8 Euler steps over the single
step graph). All 4 components are individually quantizable.
License
This conversion follows the original Supertone/supertonic-3 license
(OpenRAIL). See LICENSE (or the upstream model card).
Why fp16 and not INT4?
We attempted to ship an INT4 variant. After exhaustive testing
(INT4 sweep notes below), the supertonic-3
architecture caps at INT8 minimum for the vocoder and vector_estimator:
Component
INT4 result
Why
vocoder
cos ≈ 0 across all 6 palette configs (kmeans/uniform, pgc with group_size 8/16/32/64). Mixed-precision (head-only carve-out) also broken.
HiFi-GAN-style upsampling is uniformly sensitive — INT8 (cos 0.99) is the floor.
vector_estimator
per-step cos 0.82-0.999; over 8 diffusion steps this compounds to garbled / "backwards-sounding" audio.
Flow-matching diffusion accumulates error. INT8 works (per-step cos 0.96-0.998 except short_ja_F3 which is architecturally cos 0.82 even at INT8).
duration_predictor
smallest drift was 0.11s with pt_uniform — but enough to shift L_real → bucket-leak boundary moves → pacing perceptibly breaks.
dp output sets the diffusion frame budget; any drift propagates.
text_encoder
cos 0.97 at pgc_g32 (works alone).
Conditioning quality compounds with VE drift.
The best achievable mixed config (only voc INT8, others fp16) saves
~25 MB out of 272 MB — not worth a separate variant. The fp16 build
shipped here is the final deliverable.
Companion build
The cross-platform LiteRT version is at
Reza2kn/supertonic-3-litert.
LiteRT ships INT4/INT8 + an INT8 quantized ONNX vector_estimator
(65 MB instead of 256 MB) — but LiteRT can't reproduce the
CoreML-only "bucket-leak" extension, so long prompts sound rushed on
LiteRT. Use CoreML for full quality on Apple platforms.
