MOSS voice-profile LoRAs — ten pilot voices, and the hub for the whole LoRA ecosystem
Ten LoRA adapters for laion/moss-tts-local-transformer-4.55b-voice-acting-v2. One adapter per voice, each holding a single speaker identity across the whole expressive range: 40 emotions, 57 VoiceNet dimensions, acting edge cases, character clusters, vocal bursts, English and German.
They are also the entry point to a family of MOSS adapters — emotions, character clusters, VoiceNet dimensions, vocal bursts, domain styles — that all attach to the same base model and are designed to be stacked on top of a voice. § The ecosystem is the map; § Stacking is the part nobody gets right on the first try.
Not buried at the bottom, because these change how you should use it. Full detail in § Limitations.
No human has listened to any of this in a controlled study. Every number on this page — speaker similarity, reward, genuineness, blend, emotion strength — is the output of a learned scorer. They have been observed disagreeing with listening judgements. Treat them as relative signals between arms, never as absolute quality.
43.4 % of the training corpus falls below the 0.40 speaker-similarity floor — 57.1 % in the intense-emotion block. Identity is ranked, not gated, so the adapters were trained on takes that partly drift off the reference voice. The adapters improve this substantially; they do not remove it.
Identity is bought with expressiveness. Across the pilot the adapters raise speaker similarity far above the base model and lower genuineness. If your application values spontaneity over identity, the base model may be the better starting point.
The reference dataset is superseded-in-waiting. It encodes run PPILOT2, which has two measured text defects: 99.96 % of its 135,630 burst tags are Title-Case (MOSS spells (Growl) out letter by letter instead of performing it), and burst density is 33.7 % against 50 % intended. Both are fixed in the in-flight 500-voice build, whose indices 490–499 will replace these same ten voices.
The ecosystem: what attaches to what
Everything below is a PEFT LoRA adapter for one base model. They differ only in what they were trained to move.
the dataset these adapters were trained and evaluated on, plus every reference clip
—
pilot/<voice>/
—
The λ column is not decoration. See § Stacking — activating an adapter in peft applies it at its trained strength, which for most of these is the wrong dose.
Quickstart
Environment
Python 3.11, one CUDA GPU. The bf16 base is ~9.1 GB of weights and the audio tokenizer and KV cache sit on top of it; 24 GB is a safe floor for single-sentence generation. Tested on one GH200 (96 GB). Exactly the versions it was tested with:
trust_remote_code=True is required — the MOSS TTS modelling code ships in the base-model repo, not in transformers.
The code
Also in this repo as quickstart.py, which is the file that was actually executed.
python
1import numpy as np, soundfile as sf, torch
2from huggingface_hub import hf_hub_download, snapshot_download
3from peft import PeftModel
4from transformers import AutoModel, AutoProcessor
56BASE ="laion/moss-tts-local-transformer-4.55b-voice-acting-v2"7CODEC ="OpenMOSS-Team/MOSS-Audio-Tokenizer-v2"8LORAS ="TTS-AGI/moss-voice-profile-loras"9REFS ="TTS-AGI/moss-voice-profile-references"10VOICE ="k325_age3_bg1"# Velvet Sage Baritone11GEN_SR =48000# what proc.decode() returns1213# 1. processor + codec14proc = AutoProcessor.from_pretrained(BASE, trust_remote_code=True, codec_path=CODEC)15proc.audio_tokenizer = proc.audio_tokenizer.to("cuda").eval()1617# 2. frozen base (the "weights not initialised: audio_lm_heads/text_lm_head" warnings are benign)18model = AutoModel.from_pretrained(BASE, trust_remote_code=True, dtype=torch.bfloat16,19 attn_implementation="sdpa").to("cuda").eval()2021# 3. the voice adapter. The top level of a voice folder is the SHIPPED adapter.22# Online, `PeftModel.from_pretrained(model, LORAS, subfolder=VOICE)` is enough. Resolving23# to a local directory first also works on an air-gapped machine -- see the note below.24root = snapshot_download(LORAS, allow_patterns=[f"{VOICE}/adapter_*"])25model = PeftModel.from_pretrained(model,f"{root}/{VOICE}").eval()2627# 4. the reference clip. The adapter carries the identity, but the base model is STILL28# reference-conditioned: generate without one and you get a random speaker wearing the29# adapter. Use the same reference the adapter was trained against.30ref = hf_hub_download(REFS,f"pilot/{VOICE}/reference.wav", repo_type="dataset")3132text =("I have read the file. There is nothing in it that surprises me, "33"and that is exactly what worries me.")34instruction =("A warm, aged baritone, unhurried and contemplative, "35"speaking just above a murmur.")3637conv =[[proc.build_user_message(text=text, instruction=instruction, language="English",38 reference=[ref], tokens=max(8,len(text.split())))]]39batch = proc(conv, mode="generation")4041torch.manual_seed(0)42out = model.generate(input_ids=batch["input_ids"].cuda(),43 attention_mask=batch["attention_mask"].cuda(),44 max_new_frames=400, do_sample=True,45 text_temperature=0.7, text_top_k=50, text_top_p=1.0,46 audio_temperature=1.0, audio_top_p=0.95, audio_top_k=30,47 audio_repetition_penalty=1.1)4849# 5. ALWAYS check audio_codes_list. An empty decode is a normal silent failure of this50# model, not an exception -- on one run every candidate came back empty.51msg = proc.decode(out)[0]52assert msg.audio_codes_list,"empty decode; retry with another seed"53w = msg.audio_codes_list[0].cpu().float().numpy()54w = np.ascontiguousarray(w.mean(0)if w.ndim >1else w)55sf.write("quickstart.wav", w, GEN_SR)56print(f"{len(w)/GEN_SR:.2f}s @ {GEN_SR} Hz mono")
peft subfolder= is broken offline. In peft 0.20.0, PeftModel.from_pretrained(..., subfolder="X") works normally when the hub is reachable, but under HF_HUB_OFFLINE=1 it takes a different code path (load_peft_weights) that puts the subfolder into the filenameand passes it again as a hf_hub_download kwarg. It then looks for X/X/adapter_model.safetensors, does not find it, and raises LocalEntryNotFoundError: Cannot find the requested files in the disk cache — which reads like a missing download rather than a doubled path. The config loads fine (PeftConfig.from_pretrained handles the subfolder correctly), so only the weights fail. Resolving the repo to a local directory first, as above, avoids the branch entirely and behaves identically online and offline.
Expected output
This was run.quickstart.py was executed end to end on one GH200 (Slurm job pvdoctest, seed 0, the default text above) with the versions pinned above, and it produced:
[66s] base loaded
[67s] adapter k325_age3_bg1 attached (r=4)
[81s] wrote quickstart.wav: 5.04s @ 48000 Hz mono, peak 0.824
what to check
value
sample rate
48 000 Hz, mono — this is what proc.decode() returns. The scorers in this project resample to 16 kHz; the audio itself is 48 k.
duration
5.04 s for the 20-word default line (~4.0 words/s)
file size
473 KB (483,884 bytes) as 16-bit PCM WAV — ≈ 96 KB per second of audio
peak / RMS
peak 0.824, RMS 0.107, 8.8 % of samples near-silent
wall clock
66 s to load the base from a cold shared filesystem, ~1 s to attach the adapter, 14 s to generate and decode
If your file is a few hundred bytes, or the duration is ~0.1 s, the decode came back empty — regenerate with a different seed. If it is 5 s of noise, check that you passed reference=[...].
The decoder returns float and can exceed ±1.0 (a base-model take in the stacking run below peaked at 1.047), which soundfile silently clips when it writes 16-bit PCM. If that matters, write float (subtype="FLOAT") or normalise before writing.
The exact duration and peak will not reproduce bit-for-bit on different hardware or library versions (sampling is stochastic even at a fixed seed once the kernel schedule changes), but the sample rate, the rough length and the byte-per-second ratio will.
The three parameters that matter
parameter
what it does
notes
text
the words, and the direction
Inline tags in round brackets are directions, not spoken words: (sobs), (sharp inhale). Square brackets are pauses. Lower-case them — MOSS spells a capitalised token out letter by letter, so (Growl) is delivered "gee-are-oh-doubleyou-el". This is the single most common mistake with this model and it is what damaged the reference dataset.
instruction
the caption: who is speaking and how
Free text. The adapters were trained with captions resampled every epoch from measured attributes, so they are robust to phrasing — but the caption still steers.
reference
list of paths to the conditioning clip(s)
Required. Use pilot/<voice>/reference.wav from the dataset repo.
max_new_frames=400 is roughly a 32 s ceiling; the sampling defaults above (audio_temperature=1.0, top_p=0.95, top_k=30, repetition_penalty=1.1) are the corpus defaults and are a reasonable starting point for all ten voices.
The ten voices
Numbers are the shipped adapter measured on held-out groups it never saw (192 paired clips each, same prompts and seeds as every other arm). reference is a path inside TTS-AGI/moss-voice-profile-references; each voice folder there also holds voice.json (the identity card), metadata.parquet (83 annotation columns) and five WebDataset shards of every take generated for it.
voice (subfolder)
name
ships
params
spk-sim
base
reward
genuineness
blend
WER
reference
anime_088
Breathless Exile's Whisper
r4
8.6 M
0.4566
0.3286
7.06
0.89
5.90
0.086
pilot/anime_088/reference.wav
emolia_c0542
Measured Slavic Historian
r8
17.2 M
0.6167
0.3922
2.53
0.58
1.09
0.086
pilot/emolia_c0542/reference.wav
emolia_c1682
Cynical Streetwise Youth
r4
8.6 M
0.6083
0.3872
3.31
1.23
1.38
0.105
pilot/emolia_c1682/reference.wav
emolia_c1699
Poised Intellectual Professional
r4
8.6 M
0.5480
0.3659
4.48
0.97
2.63
0.075
pilot/emolia_c1699/reference.wav
emolia_c2570
Scholarly Matriarch Historian
r4
8.6 M
0.6123
0.4447
3.11
1.32
1.22
0.117
pilot/emolia_c2570/reference.wav
k10_age3_bg1
The Serene Storyteller
r4
8.6 M
0.6374
0.4066
3.11
0.37
1.94
0.076
pilot/k10_age3_bg1/reference.wav
k325_age3_bg1
Velvet Sage Baritone
r4
8.6 M
0.6369
0.4134
4.02
0.43
2.84
0.080
pilot/k325_age3_bg1/reference.wav
k395_age3_bg1
Fragile Matriarch
r4
8.6 M
0.5915
0.3105
6.76
1.10
5.45
0.105
pilot/k395_age3_bg1/reference.wav
k91_age5_bg0
Fading Elder Prophetess
r4
8.6 M
0.6574
0.3753
5.39
0.47
4.37
0.068
pilot/k91_age5_bg0/reference.wav
mediathek_0184
Whispering Teutonic Chronicler
r4
8.6 M
0.6880
0.5139
5.14
0.46
4.15
0.089
pilot/mediathek_0184/reference.wav
Identity cards, in the same order:
voice
gender
age
language / accent
source pool
anime_088
Male
Late 30s–40s
English, Japanese-accented
anime
emolia_c0542
Male
Late 40s–50s
English, Slavic-accented
emolia
emolia_c1682
Male
Early–mid 20s
English, AAVE
emolia
emolia_c1699
Female
Late 20s–mid 30s
English, Standard American
emolia
emolia_c2570
Female
Late 60s–70s
English, Standard American with AAVE undercurrents
emolia
k10_age3_bg1
Androgynous
Adult (30s–50s)
—
character cluster
k325_age3_bg1
Male
Late 40s–60s
—
character cluster
k395_age3_bg1
Female
Late 40s–60s
—
character cluster
k91_age5_bg0
Female
Late 70s–80s
—
character cluster
mediathek_0184
Male
Late 60s–70s
German, Standard German
German broadcast
Reading the numbers.spk-sim is ECAPA cosine to the reference clip (higher = more the same person; 0.40 is this project's floor). base is the same measurement with no adapter — the column that shows what the adapter bought. reward is the corpus's own composite ranking score and is not comparable across voices, only across arms of the same voice. genuineness (0–6) and blend (0–10) are learned heads. WER is Whisper-large-v3-turbo on the generated audio with inline tags stripped.
anime_088 is the hard case: a breathy, gravelly, heavily-accented voice whose base similarity is the lowest of the ten (0.3286) and whose adapted similarity (0.4566) is barely above the 0.40 floor. It is also the voice with 80.4 % of its corpus takes below the floor. Expect identity drift.
The non-obvious part, and the reason this page exists.
Why you can't just call set_adapter
A voice LoRA gives you who. An emotion LoRA gives you how it feels. A vocal-burst LoRA gives you the sob in the middle of the line. You want all three at once, at different strengths — and peft has no dose parameter.
PeftModel.set_adapter(name) activates one adapter at its trained scaling (alpha / r). PeftModel.base_model.set_adapter([a, b, c]) activates several, still each at its trained scaling. There is no λ argument anywhere. The dose lives one level down, in LoraLayer.scaling[name], and a stack is therefore always the same two moves:
activate the set through pm.base_model.set_adapter([...])
rewrite m.scaling[name] on every LoraLayer, relative to the trained value
The peft trap that killed four runs
PeftModel.active_adapter is a plain attribute, not a property. base_model.set_adapter() does not update it.
peft sets PeftModel.active_adapter once in __init__, to the name of the first adapter ever loaded, and thereafter only updates it inside PeftModel.set_adapter(). Step 1 above goes through pm.base_model.set_adapter() — which updates the LoraModel and leaves the PeftModel attribute pinned to the first adapter, forever.
That is harmless right up until the first adapter is deleted or evicted. From that moment every generate() evaluates peft_config[self.active_adapter] and raises:
KeyError: 'A_emo_Fear'
permanently, on an adapter name that no longer exists. It killed four sweep arms, four separate times, each stopping at exactly the group where the resident adapter set first exceeded the cache limit. Two attempted fixes missed it because peft's delete_adapter()does try to repair active_adapter — but only when exactly one adapter is left active, which is never true for a stack.
The fix is one line, after activating the set:
python
1pm.base_model.set_adapter(names)# names = the list you want active2pm.active_adapter = names[0]# the attribute peft actually indexes
The second trap: a in m.scaling is not enough
Rewriting m.scaling[a] is destructive, so you cannot read the trained value back out of it later — you have to snapshot m.scaling first and always compute the dose as snapshot × λ. The trap is that the snapshot goes stale: it is taken over the modules and adapters that exist at the moment you take it, and any adapter loaded afterwards — or reloaded after an LRU eviction — is present in m.scaling but absent from your snapshot. The naive if a in m.scaling: m.scaling[a] = base_scaling[nm][a] * s then raises KeyError on an adapter that is loaded and is active, which reads as impossible.
So: check both dicts, and when only m.scaling has the key, treat its current value as the trained one and record it. Re-snapshot after every load_adapter as well (the load() above does).
The same failure would also be produced by adapters that target genuinely different module sets. Worth knowing that in this ecosystem they do not: the voice, emotion, vocal-burst, VoiceNet-dimension, character and Mediathek adapters all declare the same 23 target-module patterns (q,k,v,o,gate,up,down_proj, c_attn, c_proj, fc_in, fc_out, audio_lm_heads.0–11) and differ only in rank — 4, 32 and 64 respectively. Checked against the published adapter_config.json of each. If you bring in an adapter from outside this family, check its target_modules before assuming.
The working pattern
Complete, runnable version in this repo as stack_adapters.py — it also generates base / voice / voice+emotion / voice+emotion+burst and then deletes an adapter and generates again, which is where trap 1 fires.
python
1from huggingface_hub import snapshot_download
2from peft import PeftModel
3from peft.tuners.lora import LoraLayer
45classAdapterStack:6"""Several LoRAs on one frozen base, each at its own merge weight."""78def__init__(self, base_model):9 self.pm =None10 self.base = base_model
11 self.base_scaling ={}# module -> {adapter: trained scaling}1213defload(self, name, model_id, subfolder=None):14 path = model_id
15if subfolder:# see the offline note in § Quickstart16 path =f"{snapshot_download(model_id, allow_patterns=[subfolder +'/adapter_*'])}/{subfolder}"17if self.pm isNone:18 self.pm = PeftModel.from_pretrained(self.base, path, adapter_name=name).eval()19else:20 self.pm.load_adapter(path, adapter_name=name)21# Re-snapshot: a module first reached by adapter B has no entry for B in a22# snapshot taken while only A was loaded.23 self.base_scaling ={nm:dict(m.scaling)for nm, m in self.pm.named_modules()24ifisinstance(m, LoraLayer)}25return self
2627defset_active(self, spec):28"""spec: {name: lambda}. Weight 0 or absent = off; {} = pure base model."""29 pm = self.pm
30 keys ={n:float(v)for n, v in spec.items()if v}31ifnot keys:32 pm.base_model.disable_adapter_layers()33return34 pm.base_model.enable_adapter_layers()35 pm.base_model.set_adapter(list(keys))3637# TRAP 1 -- see above. One line, four dead runs.38 first =next(iter(keys))39if first ingetattr(pm,"peft_config",{}):40 pm.active_adapter = first
4142# TRAP 2 -- check BOTH dicts, and record the trained value on first sight.43for nm, m in pm.named_modules():44ifnotisinstance(m, LoraLayer):45continue46for a, s in keys.items():47if a notin m.scaling:48continue49if a in self.base_scaling.get(nm,{}):50 m.scaling[a]= self.base_scaling[nm][a]* s
51else:52 self.base_scaling.setdefault(nm,{})[a]= m.scaling[a]53 m.scaling[a]= m.scaling[a]* s
545556stack =(AdapterStack(model)57.load("voice","TTS-AGI/moss-voice-profile-loras", subfolder="k325_age3_bg1")58.load("emotion","TTS-AGI/moss-emotion-loras-v3", subfolder="Anger")59.load("burst","laion/vocal-burst-lora-adapters", subfolder="sobs"))6061stack.set_active({"voice":1.0,"emotion":0.25,"burst":0.5})62# ... then generate() exactly as in the quickstart, on stack.pm
This was run too.stack_adapters.py was executed on one GH200, same seed for every arm, same sentence except where a burst tag is added:
Note arm 02 against arm 03: the capped emotion dose (0.25, from the burst rule) keeps the line intact at 3.36 s, while the uncapped intense dose (1.9) stacked on the voice adapter cuts it to 1.68 s. Also note that all three adapters declare the same 23 target-module patterns and differ only in rank — 4 for the voice, 32 for the other two.
And the trap was reproduced, on purpose. The script then deletes the first-loaded adapter — the exact trigger — and prints what peft leaves behind:
active_adapter before delete: 'voice'
active_adapter after deleting 'voice': 'voice' (peft_config now holds ['burst', 'emotion'])
^ DANGLING. Without the one-line repair in set_active(), the next generate() raises
KeyError from inside peft_config[self.active_adapter].
04_after_deleting_first_adapter {'emotion': 0.25, 'burst': 0.5} 3.60s peak 0.801
active_adapter after set_active: 'emotion'
PeftModel.active_adapter is still 'voice' after delete_adapter('voice') — a pointer to a name that is no longer in peft_config. Every subsequent generate() would raise KeyError: 'voice'. set_active() repairs it on the next call and generation proceeds normally. That is the whole bug, in four lines of output.
Both runs are bit-reproducible: the same arms on two different nodes produced identical durations and peaks.
Choosing λ
Measured, not guessed. Sources are the recipe pages linked at the bottom.
adapter
λ
why
voice (this repo)
1.0
it is the identity; the ablation below is about rank, not dose
emotion, moderate
0.5
flat across all 40
emotion, intense
0.5 – 1.9, per emotion
measured per emotion. e.g. Awe 0.5, Sadness 1.25, Fear 1.75, Anger 1.9. Using 1.9 for Awe overdrives it.
emotion, on a line that also carries a burst
≤ 0.5 × burst λ
see below
vocal burst
0.5
measured optimum for a mid-utterance burst: presence 50.3 %, tail coverage 0.90
VoiceNet dimension
per dimension and direction
best dose measured per (dim, direction) in 0.25–1.25; ×0.40 for a "somewhat/notably" step instead of an extreme. Some are traces: vn_VULN__low wants 0.06–0.18, not a full dose
mediathek (German broadcast)
0.25
sports commentator
1.0
explicitness
0.8
character archetype
1.0
the character is the condition in that block
The burst/emotion interaction is the one measured conflict on record. At burst λ = 0.5:
emotion λ
burst presence
blend
genuineness
0.00
0.505
4.76
1.89
0.25
0.569
4.90
1.91
0.50
0.441
4.45
1.57
Emotion at half the burst dose beats both dropping it and matching it, on all three metrics. So the rule is a cap, not a set: λ_emotion = min(λ_emotion, 0.5 × λ_burst). A condition already below the cap keeps its own smaller dose.
Five burst classes (hiss, kissing_noises, lip_smack, person_whistling_playfully, slurping_noises) produce no located burst at any dose when asked for mid-utterance; generate them as isolated events instead.
What happens when adapters conflict
Two adapters pulling the same modules compound, and the bigger one wins. Every family here targets the same 23 module patterns, so a voice LoRA and an emotion LoRA are always fighting over the same weights. The voice adapters are rank 4; the emotion, burst, VoiceNet and character adapters are rank 32 and Mediathek is rank 64. At equal λ the larger adapter dominates and identity drifts. That asymmetry — not any target-module difference — is why every non-voice λ in the table above is below 1 unless the adapter is the condition.
Overdriving truncates. In the stacking run below, the same sentence and seed produced 3.84 s with the voice adapter alone, 3.36 s with voice + emotion capped at 0.25 + burst 0.5, and 1.68 s — under half — with the emotion adapter at its intense dose of 1.9 on top of the voice adapter. An overdriven stack does not merely sound wrong; it stops early. (One sentence, one seed: an illustration, not a measurement.)
Order does not matter, dose does. LoRA deltas are additive; set_adapter([a, b]) is symmetric. Only the scalings differentiate them.
Keep the resident set bounded. Loading many adapters costs VRAM. Evict LRU — and re-read trap 1, because eviction is exactly what turns the latent bug into a KeyError.
Sort your work by adapter. With 114 dimension adapters and 40 emotion adapters, thrashing the loader between every generation is the easiest way to waste a GPU-hour.
The rank ablation, in full
The headline: rank 4 is enough
9 of the ten voices ship a rank-4 adapter — 8.6 M trainable parameters, a quarter of rank 16's 34.4 M and an eighth of the rank 32 the single-voice predecessor shipped. emolia_c0542 ships rank 8 (17.2 M) because rank 4 failed the non-inferiority test for that voice.
Each voice was trained at rank 16, 8 and 4 and the three were compared on held-out groups. The shipped rank is the smallest rank that is not significantly worse than the best rank on speaker similarity (paired t, p ≥ 0.05, and no more than 0.03 absolute WER worse). Speaker similarity is the primary axis because these are identity adapters; an argmax on a noisy mean would have answered "16 always" by construction.
Pooled: n = 1,920 held-out clips per arm
Every arm generated the same prompts with the same seeds; only the adapter differs.
arm
clips
spk-sim
reward
genuineness
blend
WER
emotion strength
base
1920
0.3938
3.990
0.866
2.58
0.1297
1.500
stage1_r16
1920
0.5933
4.235
0.747
2.89
0.0971
1.462
stage1_r8
1920
0.5976
4.213
0.765
2.87
0.0937
1.461
stage1_r4
1920
0.5944
4.257
0.753
2.89
0.0883
1.465
stage2_r16
1920
0.6017
4.496
0.799
3.09
0.0926
1.456
stage2_r8
1920
0.6034
4.453
0.809
3.08
0.0934
1.459
stage2_r4
1920
0.6040
4.499
0.785
3.11
0.0893
1.462
The two effects, side by side
comparison
Δ spk-sim
95 % CI
p
base → stage2_r4
+0.2102
[+0.2030, +0.2175]
< 1e-300
stage2_r16 → stage2_r4
+0.0023
[-0.0014, +0.0060]
0.22
stage2_r16 → stage2_r8
+0.0017
[-0.0018, +0.0052]
0.35
stage2_r16 → stage1_r16
-0.0084
[-0.0123, -0.0046]
1.8e-5
The base→r4 effect is ~90× the size of the r16→r4 gap, and the inter-rank gaps are indistinguishable from zero on every axis measured (reward +0.003, p = 0.94; WER -0.003, p = 0.32). Meanwhile stage 2 of the curriculum is worth something: +0.0084 spk-sim (p = 1.8e-5) and +0.26 reward (p = 1.7e-8) over stage 1.
So the rank knob is not where the quality is. Training these at rank 16 would have cost 4× the adapter parameters to buy nothing measurable.
Per-voice decisions
voice
ships
best rank
spk-sim @ ships
@ best
Δ
p
WER
anime_088
r4
r16
0.4566
0.4623
-0.0058
0.542
0.0860
emolia_c0542
r8
r8
0.6167
0.6167
+0.0000
1.000
0.0864
emolia_c1682
r4
r4
0.6083
0.6083
+0.0000
1.000
0.1052
emolia_c1699
r4
r4
0.5480
0.5480
+0.0000
1.000
0.0745
emolia_c2570
r4
r4
0.6123
0.6123
+0.0000
1.000
0.1172
k10_age3_bg1
r4
r4
0.6374
0.6374
+0.0000
1.000
0.0758
k325_age3_bg1
r4
r16
0.6369
0.6433
-0.0064
0.163
0.0795
k395_age3_bg1
r4
r8
0.5915
0.5998
-0.0084
0.162
0.1049
k91_age5_bg0
r4
r4
0.6574
0.6574
+0.0000
1.000
0.0684
mediathek_0184
r4
r8
0.6880
0.6951
-0.0071
0.159
0.0886
Four voices have a nominally better rank than the one they ship; in all four the gap is within noise (p ≥ 0.16), so the smaller adapter wins. emolia_c0542 is the exception: rank 4 was significantly worse and rank 8 ships.
The subtlety worth stating: validation loss would have been wrong
Held-out validation loss does separate the ranks, monotonically, for every one of the ten voices, without exception: r16 < r8 < r4. If you had run this ablation on loss alone, you would have shipped rank 16 ten times out of ten, with clean, consistent, unanimous evidence.
The size of it is the point:
value
base validation loss
3.94 – 4.20 nats
rank 16 (stage 2)
3.73 – 3.95 nats
mean base → r16 improvement
0.217 nats
mean r4 − r16 gap
+0.0084 nats
gap as a share of the gain
3.9 % (range 3.2 – 5.3 % across the ten voices)
The generation-based evaluation turns that 3.9 % into no measurable difference at all on 1,920 held-out clips per arm. The two measurements agree on the ordering and disagree only on whether the remaining gap is worth paying for.
The lesson: validation loss alone would have recommended rank 16, and it would have been wrong. A loss difference can be perfectly consistent, perfectly monotonic, replicate across ten independent training problems — and still be four times too small to matter in the artefact you actually ship. Ablate on the output, not on the objective. (And note the honest limit of that claim: the generation evaluation is itself a learned scorer, so what has really been shown is that the gap is below the resolution of every instrument that was pointed at it. No human was asked.)
Repo layout
<voice>/adapter_model.safetensors SHIPPED the adapter to use
<voice>/adapter_config.json SHIPPED rank, alpha = 2r, target modules
<voice>/RECOMMENDED.json SHIPPED which rank and stage, and why
<voice>/holdout_gids.json the groups this adapter never saw
<voice>/ranks/r16/{stage1,stage2}/ audit every rank, both curriculum stages
<voice>/ranks/r8/{stage1,stage2}/ audit
<voice>/ranks/r4/{stage1,stage2}/ audit
ablation/pooled.csv the pooled arm table above
ablation/per_voice_arm.csv 10 voices × 7 arms
ablation/decisions.json the shipped rank per voice, with p-values
ablation/rank_ablation.json everything, including all paired tests
quickstart.py the tested quickstart
stack_adapters.py the tested stacking pattern
166 adapter and metadata files, 5.19 GB, plus this card and the two scripts. Use the top level of a voice folder. The ranks/ tree is kept so the comparison can be redone or a different rank chosen deliberately — it is not a menu of recommendations.
Training
Two-stage curriculum, identical for all ten voices and all three ranks:
stage 1
stage 2
pool
the better half of every group's candidates by the group's own reward ranking
the winners only (rank == 0 per (gid, subset))
rows
~18,800 per voice
~1,580 per voice
peak LR
2e-4
5e-5 (0.25×)
epochs
1
1
effective batch
16 (micro 4 × accum 4)
16
schedule
linear warmup 5 % then linear decay to 0
warmup 10 %
AdamW (weight_decay=0, betas 0.9/0.999), grad-norm clip 1.0, max 380 codec frames (~30 s), seed 42, bf16, frozen base. LoRA alpha = 2r, dropout 0.05, bias=none, targeting the global Qwen3 stack (q,k,v,o,gate,up,down_proj), the local GPT-2 decoder (c_attn, c_proj, fc_in, fc_out) and all twelve audio heads (audio_lm_heads.0..11).
The three ranks are trained in one process against one shared frozen base, stepping on the same micro-batches with the same seed. Rank is then the only difference between the three adapters — same data order, same sampled captions, same dropout draws — which is what makes the paired comparison meaningful.
Captions are resampled every epoch from the stored measurements, seeded by (uid, epoch). The dataset's own caption_gen column is not used for training: it reproduces a known-broken generator that slices the first nine words off paragraph-long VoiceNet anchors. The training caption is rebuilt from the same measurements with varied skeletons, synonyms and dimension subsets, and with probability 0.35 the group's authored caption is used instead — so the adapter sees both the kind of prompt a user writes and the kind a scorer produces.
Held-out groups are removed before encoding, not at training time, so no held-out audio is ever in the training container. The split is by group and stratified by (block, language): the candidates of a group are the same condition, so holding out candidates rather than groups would leak the line.
Compute: 38.8 GPU-hours on GH200s (698 core-hours) for all ten voices — encoding, three ranks × two stages each, and the held-out evaluation.
Limitations, in full
No human listening study was run. Every number on this page is an automatic scorer's output: speaker similarity is ECAPA cosine (speechbrain/spkrec-ecapa-voxceleb); genuineness, blend and emotion strength are learned heads. They have been observed disagreeing with listening judgements. Nothing here has been validated against human preference.
43.4 % of the training corpus falls below the 0.40 speaker-similarity floor. By block:
block
takes
spk-sim
% below 0.40
voicenet
218,880
0.434
40.1
emotion
153,600
0.411
46.8
— of which intense, free
38,400
0.363
57.1
— of which intense, contained
38,400
0.378
52.4
— of which moderate
76,800
0.452
38.7–39.0
edge cases
13,440
0.393
49.6
character
11,520
0.387
52.1
burst isolated
3,200
0.451
36.7
explicit
960
0.398
50.5
sports
960
0.295
78.4
all
402,560
0.422
43.4
Identity is ranked at 0.19 weight, not gated — a hard 0.68 gate was rejected because 55 % of genuinely same-speaker pairs fall below it. The consequence is that the adapters were trained on takes that partly drift off the reference. They improve on it (pooled 0.3938 → 0.6040 on held-out clips) but the intense-emotion conditions remain the hardest place to hold a voice, and sports is worse still.
Identity is bought with expressiveness. The adapters raise speaker similarity and lower genuineness — the same trade the single-voice predecessor measured (genuineness 0.784 → 0.581 at rank 16). If spontaneity matters more than identity to you, start from the base model.
The reference dataset is superseded-in-waiting. It encodes run PPILOT2 and carries two measured text defects:
defect
intended
measured
how measured
burst tags Title-Case — MOSS spells (Growl) out letter by letter instead of performing it
lower-case (growl)
135,580 of 135,630 tags (99.96 %) are Title-Case; median 13,544 per voice
counting \(…\) spans in the released text column
burst density
50 % of lines carry a burst
33.7 % of rows do
same
Both are fixed in the in-flight 500-voice build. Its indices 490–499 are these same ten voices and will replace them. Nothing here is wrong about rank — that conclusion is independent of the tag casing — but the audio itself will be redone.
WER on this page is Whisper-large-v3-turbo, not the Parakeet used for the dataset's own wer column: the eval process already holds the generator, the scorer and ECAPA, and NeMo cannot be imported beside them. These numbers are comparable across arms, which is what a rank comparison needs, but not to the dataset's column.
Two languages (English, German). Synthetic training data throughout — these adapters were trained on MOSS output, scored by learned heads, not on recorded human speech.
anime_088 is weak. Adapted spk-sim 0.4566 is barely above the floor, and 80.4 % of its corpus takes are below it. Use it knowing that.
1@misc{moss_voice_profile_loras_2026,
2 title = {MOSS voice-profile LoRAs: ten pilot voices and a rank ablation},
3 author = {LAION and TTS-AGI},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/TTS-AGI/moss-voice-profile-loras}}
6}
