redline-v1
AI timing for osu! — a 45M-parameter frame-level model that turns raw audio into
osu! timing (redlines: BPM / offset / meter), built for the songs that are hard to time
by hand: live drummers, metal, rubato, accelerandos. Output is a starting point to
verify by ear.
Model
Encoder-only transformer (rotary attention, conv frontend over log-mel + onset flux),
per-frame outputs at 50 fps:
| head | output |
|---|
| beat | beat-here logit |
| down | downbeat (measure start) logit |
| off | sub-frame offset (recovers ~2-5 ms precision from 20 ms frames) |
| tempo | BPM-bin logits (per-frame tempo curve) |
| change | major (audible) tempo/meter change logit |
| red | notation logit — where a human mapper writes a redline, including convention re-anchors |
d_model 512, depth 14, 8 heads, FFN 2048. Trained on ranked osu! beatmaps
(three-stage lineage; configs in the repo) on a single RTX 3080.
Results (held-out data)
Ranked mapsets excluded from training by song-level dedup, scored against the ranked map's
human timing.
Model heads, 1000 held-out mapsets (frame-level, ±40 ms):
| genre | maps | beat F1 | downbeat F1 | offset MAE |
|---|
| Pop | 285 | 0.966 | 0.885 | 4.7 ms |
| Anime | 89 | 0.969 | 0.895 | 5.2 ms |
| Rock | 192 | 0.963 | 0.863 | 4.8 ms |
| Electronic | 224 | 0.962 | 0.886 | 5.1 ms |
| Metal | 50 | 0.947 | 0.855 | 5.2 ms |
| Video game | 100 | 0.944 | 0.863 | 5.1 ms |
| other | 60 | 0.916 | 0.801 | 5.6 ms |
| all | 1000 | 0.959 | 0.873 | 5.0 ms |
End-to-end (audio → redlines), 366 held-out mapsets, ~50+ per genre:
| genre | maps | beats within 10 ms | typical map | redline match P / R |
|---|
| Pop | 55 | 89.3 % | 100.0 % | 0.51 / 0.54 |
| Metal | 48 | 88.2 % | 96.9 % | 0.55 / 0.43 |
| Rock | 51 | 87.3 % | 95.2 % | 0.42 / 0.42 |
| Electronic | 49 | 85.0 % | 100.0 % | 0.57 / 0.57 |
| Video game | 54 | 82.7 % | 98.7 % | 0.47 / 0.50 |
| Anime | 55 | 80.9 % | 98.8 % | 0.65 / 0.50 |
| other | 54 | 80.2 % | 92.7 % | 0.40 / 0.52 |
| all | 366 | 84.7 % | 98.7 % | 0.51 / 0.50 |
Genre gaps are mostly sample size: on the 1000-map run the bootstrap intervals overlap
(metal − anime = +3.2 pt, 95 % CI [−4.1, +10.5]). Read the "all" row.
The typical song comes out on-grid (median 93–100 % per genre; 240 of 366 maps above 90 %).
Of the 185 songs a human timed with one redline, the model writes exactly one on 147.
Known failure modes
~7 % of maps score below 50 %. Classifying those by the shape of the timing error gives
four distinct causes:
- The reference is offset, not the model (~27 %). A perfectly constant displacement,
~12 ms, so a 10 ms tolerance scores ~0 % while the grid is ~2 ms from right. Checked
against the audio itself (spectral-flux onsets, detector latency calibrated out): on
19 of 20 such maps the model is 2.0 ms from the audio-true beat and the mapper 16.0 ms
away — outside tolerance from their own song. Not an MP3-delay artifact.
- Metrical-level disagreement (~17 %). ×2, ÷2 or ×2⁄3 of the mapper's BPM — the same
pulse at a different metrical level, a notation convention rather than an acoustic fact.
- Genuinely hard audio (~40 %). Live recordings, solo-piano rubato, jazz swing. Beat
activation drops to 0.650 vs 0.942 on healthy maps; neither grid sits on strong onsets.
- Odd tempo ramps (~4 %). Too few maps to conclude anything.
Excluding only the proven-offset maps moves the 1000-map benchmark from 88.5 % to 90.2 %
and the failure rate from 7.0 % to 5.1 %. The reference set is not hand-cleaned, so numbers
are reported as measured.
Usage
1pip install -r requirements.txt # from the repo
2hf download Tiger14n/redline-v1 redline-v1.pt --local-dir models/
3python -m redline.generate_timing song.mp3 --osz
The checkpoint contains model (state dict) and args (data representation + architecture),
which redline.infer.Model2Beats uses to reconstruct the exact training-time pipeline.
