Jalak VITS — Sundanese/Javanese/Indonesian multi-speaker TTS (fine-tuned)
A Coqui VITS multi-speaker model fine-tuned from the
jalak-model pretrained checkpoint
(83 speakers)
+ 41 new Sundanese speakers from
OpenSLR SLR44,
for a total of
124 speakers. Input is
IPA phonemes (espeak-ng
id voice).
Quality (honestly reported)
Eval mel-reconstruction loss on a held-out 525-clip split (original speakers only,
never trained on), measured by an independent evaluator (eval_model.py):
| Model | eval_mel_loss | eval_kl | notes |
|---|
| Pretrained (jalak) | 22.73 | 4.30 | baseline |
| This model (124 spk, +SLR44) | 21.19 | 2.72 | −6.8% mel, −37% KL |
| 83-spk variant (no SLR44) | 21.19 | 2.61 | tied on metric; fewer speakers |
Adding SLR44 data was neutral on the original-speaker mel metric (21.19 ≈ 21.19,
within the ~0.15 run-to-run noise floor) but adds 41 new Sundanese speakers the
pretrained model could not synthesize at all (no speaker embeddings). Both original and
new speakers produce valid, audible speech (verified via generation sanity checks).
Caveat — new speakers are less verified: the 41 SLR44 speaker embeddings were trained
for a single (diluted) epoch; their perceptual quality is not rigorously held-out-evaluated
(only confirmed non-silent/non-garbage). The original 83 speakers are well-trained and
held-out-verified. If you need only the metric-best on original speakers, the 83-spk
variant is equivalent.
Training recipe
- Fine-tuned fresh from the pretrained checkpoint (pre-extended emb_g 83→124 so all
layers restore; the 41 new embeddings start from small random and train).
- 1 epoch over combined data (~16.9k clips: 12.6k original + 4.2k SLR44), batch 24.
- lr 5e-5, cosine decay 5e-5 → 1e-5 over the epoch (per-step), fp32.
freeze_encoder=True (text encoder frozen; acoustic/decoder adapt).- No clip-length cap, no speaker-weighted sampler, no grad clip.
- TTS 0.22.0, torch cu126, Tesla T4.
How to use
Requires espeak-ng installed (system), Coqui TTS, and this repo's make_config.py:
1# plain text -> IPA -> audio (original speaker)
2python inference.py --ckpt model.pth --text "halo selamat datang" \
3 --speaker SU-00060 --out out.wav
4
5# a new SLR44 Sundanese speaker
6python inference.py --ckpt model.pth --text "panonton ting haruleng ningali tipi" \
7 --speaker SLR44_suf_00297 --out slr.wav
8
9# list all 124 speakers
10python inference.py --ckpt model.pth --list-speakers
inference.py auto-detects the 124-speaker checkpoint and loads speakers.pth.
Files
model.pth — VITS checkpoint (124 speakers, step ~1260704)config.json — derived training/inference configspeakers.pth — speaker name → id map (124)inference.py — generation script
License
CC BY-SA 4.0 (inherited from OpenSLR SLR44; the base jalak pretrained model and the
original dataset licenses also apply — verify before commercial use).
Reproducibility
Full pipeline in the source repo: preprocess_slr44.py (data), make_config.py +
run_train.py (training), eval_model.py (evaluation), inference.py (inference).
