A LoRA for ACE-Step 1.5 acestep-v15-base (2B) that makes the model sing unaccompanied when nothing in the prompt told it to. The un-adapted base model, given a prompt that describes only a voice, still builds a backing track. This adapter removes most of it without touching diction.


Recommended	root of this repo — variant A, rank 128 / alpha 256, merge scale 1.0
Base model	`ACE-Step/acestep-v15-base` (2B). Not XL, not turbo, not SFT.
Effect	accompaniment fraction 0.188 → 0.084 (−55%), 28/40 clips improved, sign-test p = 0.017, n = 40
Cost	1.59 GPU-hours on one GH200 (13.93 across all 8 variants)
Demo	side-by-side: base ACE vs this LoRA vs LeVo 2 — 26 samples, three arms each. The original 120-player base-vs-LoRA grid is linked from the top of that page.

⚠️ Read this before you generate anything

ACE-Step's dcw_enabled parameter defaults to True, and that default destroys the CFG checkpoints — including the one this LoRA attaches to.

With the default left alone, a 30-second clip comes back transcribing as the single word "so". Measured against a Whisper transcript:

`dcw_enabled`	Whisper F1
`True` (library default)	0.021
`False`	0.911

Pass dcw_enabled=False in every generation call. infer.py in this repo forces it. If you wire the adapter into your own pipeline and get gibberish, this is why — the adapter is not the problem.

Usage

infer.py in this repo runs as-is and pulls the adapter from the Hub on first use.

bash

pip install acestep                # or: git clone ACE-Step-1.5 && pip install -e .
wget https://huggingface.co/TTS-AGI/acestep-vocals-lora/resolve/main/infer.py

python infer.py \
  --caption "A solo female voice sings an Andean huayno, high and keening, with a bright nasal placement, ornamented turns and a piercing sustained upper register." \
  --lyrics "[verse]. The quarry filled with rain the year I left. I watched it take the ladder and the crane." \
  --duration 30 --seed 770001 --out out.flac

Useful flags: --adapter DIR (local copy instead of the Hub), --scale 0.5 (weaker), and --no-lora (the un-adapted baseline, for A/B).

Settings — exactly what the numbers on this page were produced with


base checkpoint	`acestep-v15-base` (2B)
LoRA merge scale	1.0
inference steps	50
guidance scale (CFG)	7.0
`dcw_enabled`	`False`
lyric LM	`acestep-5Hz-lm-4B`, `lm_temperature` 0.85
`thinking` / `use_cot_metas`	`True` / `True`
`use_cot_caption` / `use_cot_language`	`False` / `False`
`vocal_language`	`en`
duration	30 s

50 steps at guidance 7.0 is the CFG-model pairing. The 8-step / guidance-1.0 pairing you will see in ACE-Step examples belongs to the turbo checkpoints and produces mush on base.

Attaching it yourself

python

from acestep.handler import AceStepHandler
dit = AceStepHandler()
dit.initialize_service(project_root=".", config_path="acestep-v15-base", device="cuda")
dit.load_lora("path/to/adapter")     # directory basename must contain NO dots (see below)
dit.set_use_lora(True)
dit.set_lora_scale(1.0)
assert dit.get_lora_status()["active"]

load_lora() uses the adapter directory basename as the PEFT adapter name, and PEFT rejects names containing dots. Stage the adapter into a dot-free directory first — infer.py does this for you. This is what makes raw training checkpoints (epoch_2_loss_0.7641) unloadable as written.

get_lora_status() returns {'loaded', 'active', 'scale'} — not lora_loaded / use_lora. Asserting the wrong keys silently disables the check, not the adapter.

Files

Path	Variant	Base	Rank/α	Notes
`adapter_model.safetensors` (root)	A	`acestep-v15-base`	128/256	✅ Recommended. Use this one, at scale 1.0.
`variants/B_base2b_r64/`	B	`acestep-v15-base`	64/128	Secondary. Half the rank, statistically indistinguishable at scale 0.5 — rank 128 buys nothing.
`variants/H_base2b_gr90/`	H	`acestep-v15-base`	128/256	Secondary. `genre_ratio 90`. Most consistent variant (30/40, p = 0.002) but a smaller effect; never measured at scale 1.0.
`variants/G_base2b_mlp/`	G	`acestep-v15-base`	128/256	Secondary. Also targets the SwiGLU MLP. Nearly 2× the file size, no gain.
`variants/F_base2b_lr5e5/`	F	`acestep-v15-base`	128/256	Secondary. LR 5e-5. Weakest variant (p = 0.43).
`variants/C_turbo2b_r128/`	C	`acestep-v15-turbo`	128/256	Secondary, different base. Only for turbo. Largest effect of any variant, and it also lifts turbo's lyric recall 0.644 → 0.781.
`variants/D_sft2b_r128/`	D	`acestep-v15-sft`	128/256	Secondary, different base. Only for SFT.

Each variant is bound to the checkpoint it was trained on — C and D will not do anything sensible on acestep-v15-base. Variant E (XL/4B base) is absent: it produced NaNs at step 70, inside warmup, and no usable weights. That says XL needs a lower LR, not that XL cannot work.

Training

Data — 1,314 clips, 60 genres

Selected from a pool of 8,582 captioned singing clips by a deliberately strict filter:

Filter	Remaining
starting pool	8,582
`solo_voice ∧ ¬backing_voices ∧ ¬instruments_audible`	2,544
∧ duration ≥ 25 s	1,973
∧ lyrics present	1,314

The last cut is the large one and it is not arbitrary: only one of the source runs carries lyrics, and training a vocal adapter on empty lyric conditioning teaches wordless singing, so those 659 rows were dropped rather than padded. The result is 1,314 clips over 60 genre labels (min 3 / median 20 / max 48 per genre), 25–30 s each, 48 kHz stereo, with [verse]/[chorus] markers and long prose captions describing the voice — register, vibrato, attack, mic distance — rather than tag lists.

genre_ratio = 0: captions are used in full. The Side-Step docs suggest genre_ratio 90 (swap the caption for a bare genre tag on 90% of samples) for style LoRAs, but the prompts this adapter is meant to fire on are ~300-character prose. Variant H tests that choice and does not overturn it. No trigger word — the adapter should improve vocals-only rendering unconditionally.

Synthetic-source caveat. The training audio is generated, not recorded: all 1,314 clips are LeVo 2 output, captioned by an audio LLM. This adapter distils LeVo 2's a cappella behaviour into ACE-Step. It learns the vocal character of a synthetic singer and inherits that generator's artefacts and biases. It is not trained on, and does not clone, any real performer.

Hyperparameters (variant A)


adapter	LoRA, rank 128 / alpha 256, dropout 0.1
target modules	`q_proj` `k_proj` `v_proj` `o_proj` (attention only, self + cross)
optimizer	adamw, LR 1e-4, cosine schedule, 100 warmup steps
weight decay / grad clip	0.01 / 1.0
batch	1 × gradient accumulation 4
schedule	12 epochs = 3,940 optimizer steps
timestep sampling	`fixed` (continuous logit-normal, μ −0.4 / σ 1.0, from the model's own `config.json`)
CFG dropout	0.15
seed	42
cost	1.59 GPU-hours, one GH200

bash

python train.py --plain -y fixed \
    --checkpoint-dir <ckpts> --model-variant acestep-v15-base \
    --dataset-dir <preprocessed-tensors> --output-dir <out> \
    --batch-size 1 --gradient-accumulation 4 \
    --optimizer-type adamw --scheduler-type cosine --warmup-steps 100 \
    --weight-decay 0.01 --max-grad-norm 1.0 --seed 42 --save-every 2 \
    --rank 128 --alpha 256 --lr 1e-4 --epochs 12 --dropout 0.1 \
    --cfg-ratio 0.15 --adapter-type lora --attention-type both

All eight variants were trained as independent single-GPU processes, four to a node — Lightning Fabric was not installed and the trainer degrades to a single-device loop with only a warning, so four different variants per node was a better use of whole-node billing than one faster run. Total 13.93 GPU-hours for all eight.

What was held out

No training clips were held out. All 1,314 went into training. Evaluation is instead on 40 prompts written specifically for it — 20 style captions × 2 lyric takes — and those captions and lyrics appear nowhere in the training corpus (verified: 0 of 20 captions and 0 of 2 lyric texts occur in the training set). So the evaluation measures generalisation to unseen prompts, not held-out audio, and it cannot detect memorisation of training clips.

Evaluation

Metric. accomp_frac = RMS(instrumental stem) / (RMS(instrumental) + RMS(vocal)) after BS-Roformer separation (ep.317, SDR 12.98). 0 = pure a cappella, 0.5 = instruments as loud as the voice. Separation artefacts are real but identical across both arms of a pair, so the trustworthy quantity is the within-pair difference, not the absolute level.

Design. 40 prompts (n = 40), each generated twice with identical caption, lyrics and seed in both arms, so every comparison is paired. p-values are two-sided exact sign tests over the 40 paired differences; CIs are 10k-resample bootstraps of the mean difference.

These prompts describe only the voice and never mention instruments. That condition matters: on the other grid, whose captions all say "no instruments are present", the un-adapted model already scores 0.02 and there is no headroom for any adapter to show anything (22/40, p = 0.64). Building the neutral grid was the difference between a null result and a measured one.

Variant A (recommended)

arm	accomp. fraction	Δ paired	improved	p (sign)	95% CI	lyric recall
un-adapted base	0.1880	—	—	—	—	0.8614
LoRA @ 1.0	0.0838	−0.1042	28/40	0.017	[−0.153, −0.057]	0.8585
LoRA @ 0.7	0.1264	−0.0616	25/40	0.154	[−0.109, −0.017]	0.8690
LoRA @ 0.5	0.1361	−0.0519	25/40	0.154	[−0.096, −0.011]	0.8760

At scale 1.0 that is a 55% relative reduction in accompaniment energy. The effect grows monotonically with scale over the range tested. Lyric word recall (Parakeet ASR) is flat — 0.861 → 0.859 — with 0 collapsed clips out of 160, so the a cappella behaviour is not bought by degrading diction.

All variants

Each is paired against the un-adapted arm of its own base checkpoint, n = 40:

variant	scale	un-adapted	with LoRA	Δ	improved	p	recall
A r128	1.0	0.1880	0.0838	−0.1042	28/40	0.017	0.861 → 0.859
A r128	0.7	0.1880	0.1264	−0.0616	25/40	0.154	→ 0.869
A r128	0.5	0.1880	0.1361	−0.0519	25/40	0.154	→ 0.876
B r64	0.5	0.1880	0.1318	−0.0562	28/40	0.017	→ 0.875
G +MLP	0.5	0.1880	0.1242	−0.0638	26/40	0.081	→ 0.874
H gr90	0.5	0.1880	0.1284	−0.0597	30/40	0.002	→ 0.857
F LR 5e-5	0.5	0.1880	0.1477	−0.0404	23/40	0.430	→ 0.867
C turbo	0.5	0.4652	0.3769	−0.0883	33/40	<1e-4	0.644 → 0.781
D SFT	0.5	0.2336	0.2064	−0.0272	28/40	0.017	0.879 → 0.870

Why A at 1.0 is the shipped choice: it has the largest paired effect of any arm on acestep-v15-base (−0.104, roughly double every scale-0.5 arm) at p = 0.017. The honest qualification is that only A was evaluated at scale 1.0 — H is the most consistent variant at 0.5 (30/40, p = 0.002) with a smaller effect, and it was never tested at 1.0, so "A is best" is a statement about the arms that were actually measured, not a proof that A dominates H.

Secondary findings: rank 128 buys nothing over 64; LR 5e-5 is the weakest setting; genre_ratio 90 did not hurt, contradicting the prediction made when the dataset was built; and un-prompted a cappella compliance runs base < SFT < turbo, so the choice of base checkpoint matters more than the adapter does.

Limitations

Style flattening at scale 1.0 is unmeasured. It is the documented failure mode of timbre LoRAs at high merge scale, and an energy-ratio metric is structurally blind to it. This is the main open risk on the "ship at 1.0" recommendation — judge it by ear on the demo page, which carries scale 1.0 next to the un-adapted baseline.
The evaluation uses 2 distinct lyric texts across its 40 clips (20 captions × 2 takes). Prompt diversity is real; lyric diversity is not.
No human listening study. Every number here is automatic (source separation + ASR).
2B only. Transfer to the XL/4B checkpoints is untested; the one XL run NaN'd in warmup.
Nothing else improved. The adapter does not improve pitch, musicality or diction — it removes accompaniment nobody asked for, and that is the whole claim.
Prompts that already say "no instruments are present" get no measurable benefit: the base model already complies there.

Provenance

Trained and evaluated on JUWELS/JUPITER GH200 nodes. Training audio is LeVo 2 output (see the synthetic-source caveat above). Apache-2.0, following the base model.