AffectScore -- LoRA r=32 (full, selected)
LoRA adapter for
ACE-Step v1 3.5B fine-tuned for real-time affect-conditioned music generation in narrative-driven games. This is the
primary checkpoint selected by held-out MER accuracy.
Part of the AffectScore system. Code and Colab notebooks:
github.com/LeeTgk/affectscore (link active after publication)
What this model does
AffectScore conditions music generation on a two-layer signal:
- Layer 1 -- Designer intent (authored per scene):
scene_valence, scene_arousal, arc_position on the Russell circumplex. Converted to mood-word text prompts (e.g. "melancholic, tense") via a lookup table.
- Layer 2 -- Player engagement (runtime behavioral):
choice_latency_norm, dwell_deviation_norm, interaction_rate_norm. Applied as an inference-time text suffix that modulates intensity and texture around the Layer 1 anchor without overriding it.
LoRA targets cross_attn.q_proj and cross_attn.v_proj only. Self-attention and FFN stay frozen.
Training
| Parameter | Value |
|---|
| Base model | ACE-Step/ACE-Step-v1-3.5B |
| LoRA rank | 32 |
| LoRA targets | cross_attn.q_proj, cross_attn.v_proj |
| use_rslora | True |
| lora_dropout | 0.05 |
| Trainable parameters | 7.86M (0.237% of total) |
| Training clips | 1,571 (CC0/CC-BY, CLAP-quality filtered) |
| Epochs | 50 |
| Learning rate | 1e-4 |
| Batch size | 8 |
| Quadrant weighting | max-quadrant-weight 5.0 (compensates Q2 scarcity) |
| Hardware | NVIDIA A100 (Google Colab) |
Evaluation
All metrics computed on 200 held-out clips (50 per V-A quadrant) at 40 diffusion steps.
Audio quality (EVAL-02)
| FAD-MERT ↓ | CLAP-score ↑ | KLD ↓ | PCE ↑ |
|---|
| 33.06 | −0.45 | 1.17 | 3.57 |
Note: FAD-MERT is higher than the no-lora baseline because LoRA fine-tuning specializes the model distribution toward the training corpus. This is expected and not evidence of degraded quality -- see the paper for full discussion.
Emotion controllability (EVAL-03)
Pearson r between designer-intended and music2emo-predicted V-A on 200 held-out clips. No variant achieves significant positive correlation (critical value |r| ≥ 0.138 at α=0.05); null result attributed to Q3-heavy training distribution (78.1% melancholic clips) and text-space ceiling.
| r_valence | RMSE_valence | r_arousal | RMSE_arousal |
|---|
| −0.171 | 0.520 | −0.087 | 0.292 |
How to use
This adapter is designed to run inside the AffectScore FastAPI server, which handles conditioning, generation, and WAV encoding end-to-end. Direct PEFT loading is possible but the conditioning pipeline is non-trivial -- refer to server/affectscore_server.py in the code repository.
1python server/affectscore_server.py \
2 --lora HiiragiLee/affectscore-ace-step-r32-20260629
The server exposes a /generate endpoint accepting a 6-float affect embedding (Layer 1 + Layer 2), style prompt, chunk duration, and step count.
Ablation variants
Citation
Citation will be added once the paper is published. If you use this model before then, please link to this repository.
