Views
No views yet
samples/laugh_demo.wav ·
samples/sigh_throat_demo.wavphoneme_ids mode with unused phoneme IDs repurposed as non-verbal marker
tokens, then trained on an emotive corpus (distilled from ChatterBox Turbo,
MIT) in which those markers align with genuine non-verbal audio. Inject a
marker at synthesis time and the model performs it, in-voice, with natural
prosody flowing through it:| Tag | Phoneme id | Marker char | Repeat |
|---|---|---|---|
[laugh] | 39 | æ | 5 |
[sigh] | 40 | ç | 3 |
[gasp] | 42 | ø | 2 |
[groan] | 43 | ħ | 3 |
[clear throat] | 45 | œ | 3 |
markers.json in this repo is the canonical map.)1echo "The quick brown fox jumps over the lazy dog." | \
2 piper -m en_US-mira-tokens2.onnx -f out.wavmira_synth.py (MIT,
included) is the reference implementation: it phonemizes the text segments,
splices the marker char (× its repeat count) where a [tag] appears, converts
to ids once, and calls phoneme_ids_to_audio:1from piper import PiperVoice
2import json
3from mira_synth import synth_expressive
4
5voice = PiperVoice.load("en_US-mira-tokens2.onnx")
6markers = json.load(open("markers.json"))
7wav = synth_expressive(voice, markers, "Oh that's funny [laugh] tell me the rest.")
8open("out.wav", "wb").write(wav)en — NOT
en-us. The shipped config already says "voice": "en". Do not change it,
and do not phonemize inputs with en-us. Two things break if you do:en-us emits æ for TRAP vowels ("half", "crash", "laugh"…) — and in
this voice æ is repurposed as the laugh token (id 39). Phonemize with
the wrong dialect and TRAP-vowel words trigger giggles mid-word or collapse
into mangled forms ("half" → "huff"-like artifacts). This is not
hypothetical; we shipped that bug to ourselves once.en-us adds diphthong symbols (aɪ aʊ ɔɪ eɪ oʊ) that map to ids this
model never trained — silent quality damage on any word containing
them.--data.espeak_voice en to the Piper trainer,
and note that some trainer versions regenerate the exported config from the
training flag — if your flag says en-us, the exported .onnx.json will
too, and the corruption becomes the model's default. Check the exported
config's "voice" field before deploying. The extra en-us symbol entries a
bad export adds (diphthongs at ids 161–165) are the fingerprint of this
mistake.en_US-ljspeech-medium checkpoint — LJSpeech dataset is
public domain.phoneme_ids mode, marker tokens as above.mira_synth.py: MIT.