An ONNX export of the speaker (tone colour) encoder from
OpenVoice v2 by MyShell.ai, exported from
the official converter/checkpoint.pth and verified against the reference implementation.
Used by Soninho Mágico to let a family enrol a loved one's voice on
device: the recording never leaves the phone, and only the resulting 256-float embedding is
kept.
Attribution
The model weights are the work of MyShell.ai, released under the MIT licence and
reproduced here in full in LICENSE. This repository contains only a format conversion —
no retraining, no fine-tuning, no change to the weights' values.
The STFT is baked into the graph — the caller passes raw samples and does no signal
processing of its own. Audio must be at 22050 Hz; the encoder does not resample, and feeding
it another rate returns a confident embedding for a pitch-shifted voice rather than an error.
python
1import numpy as np, onnxruntime as ort
23sess = ort.InferenceSession("tone_extract.onnx")4embedding = sess.run(None,{"audio": audio_22050[None,:].astype(np.float32)})[0]5# -> (1, 256, 1)
How it was exported
Two details that are easy to get wrong, recorded because they cost real time:
torch.onnx.export(..., dynamo=False). Torch 2.13's default dynamo exporter fails to
decompose the reflect padding under a dynamic axis. The legacy TorchScript path handles it.
The STFT is a conv1d Fourier basis, not torch.stft. OpenVoice already ships this as
spectrogram_torch_conv in openvoice/mel_processing.py, precisely because torch.stft
exports badly.
Cosine similarity against the reference ToneColorConverter.extract_se on the same audio:
torch wrapper vs official extract_se: cos=1.000000
ONNX vs official extract_se: cos=1.000000
Anything below ~0.999 means the graph is wrong rather than merely imprecise.
Intended use and limits
Built for consented voice enrolment in a children's bedtime-story app — a grandparent
recording their own voice so it can read stories aloud.
The MIT licence imposes no use restrictions. The following are our requirements, not the
licence's, and are stated here because anyone reusing this should think about them:
Get the speaker's consent, not the consent of whoever is holding the phone. Voice used
for identification is biometric data; under Brazil's LGPD (art. 11) it is sensitive
personal data, and children's data (art. 14) needs specific parental consent on top.
Disclose synthetic speech. Audio produced this way should not be presented as a
genuine recording of a real person.
This model only produces an embedding. It cannot synthesise speech on its own, and the
embedding cannot be inverted back into the original recording.
