✅ v2 release (17 Jul 2026): the completed second training run —
47,532 steps over 1.5M samples spanning engraved scores, lead sheets
with chords and lyrics, and pianoform music. Reads piano / grand-staff
scores, chord symbols (incl. European conventions like Em written
"em" and German H), and multi-verse lyrics. Ranked #1 on the
IMSLP Piano Scores leaderboard at 0.4409 OMR-NED (20 Jul 2026),
with less than half the error of GPT-5 and Gemini 2.5 Pro on the
same pages. Handwritten-manuscript support is the next training run.
rokot-omr-2b turns images of printed sheet music — scans, photos,
pages out of a PDF — into machine-readable notation. Show it a system of
music and it writes rokot-ABC, a strict, canonical profile of ABC
notation that converts losslessly to MusicXML. From there your score opens
directly in MuseScore, Finale, Dorico, or anything else that speaks
MusicXML — ready to edit, transpose, and play back.
Because the output is plain ABC text, the model is small enough and simple
enough to run locally on your own machine — including in
LM Studio on a Mac: load the model, drop in a
cropped image of a staff, and copy the ABC out. Standard ABC tooling
(abc2xml, EasyABC, abcjs, …) reads rokot-ABC as-is, since it is exactly
one deterministic spelling of ordinary ABC.
Note the unit of work: the model reads one system at a time (one line
of music, cropped to the staff). Turning a whole page or PDF into a single
score additionally needs page segmentation and measure-wise joining —
that's a pipeline around the model, not the model itself. LM Studio is
great for trying it out and for transcribing lines you crop yourself.
It is a full fine-tune of Qwen/Qwen3-VL-2B-Instruct
(2.1 B parameters, Apache-2.0 base), trained on a large synthetic corpus of
engraved scores rendered with realistic page degradation — so it is built for
the sheet music people actually have: photocopies, songbook scans, lead
sheets with chords and lyrics, not just clean digital engravings.
What it can read
Single staff (treble or bass) and grand-staff systems, up to 2 voices per staff
Pitches, accidentals, rests, chords, tuplets, dotted values from breve to 1/64
Lyrics with multiple verses, melisma and syllable continuation
Out of scope (for now): more than one part, percussion/TAB, nested tuplets,
and handwritten manuscripts — that last one is actively being worked on.
How it scores
On the public IMSLP Piano Scores leaderboard
— 32 real scanned piano pages from IMSLP, scored by OMR-NED (a normalized
edit distance over the full MusicXML; lower is better):
System
OMR-NED ↓
rokot-omr-2b (this model) — #1
0.4409
homr
0.9050
Gemini 2.5 Pro
0.9403
GPT-5
0.9438
Standings as of 20 Jul 2026. The score is produced by our full-page
pipeline: staff-system detection, per-system transcription with this
model, and measure-wise joining into one MusicXML per page — the same
per-system transcription you get from the quick-start below, run over
every system of a page. A 2.1 B model you can run on a laptop, leading
the board.
Quick start in LM Studio (Mac & PC)
GGUF builds are included in this repo for local use with
LM Studio or llama.cpp:
File
What it is
Size
rokot-omr-2b-Q8_0.gguf
Recommended — 8-bit, verified output matches the full model
~1.8 GB
rokot-omr-2b-f16.gguf
Full-precision text model
~3.4 GB
mmproj-rokot-omr-2b-f16.gguf
Vision projector — required alongside either model
~0.8 GB
Search for rokotmidi/rokot-omr-2b in LM Studio's Discover tab (it fetches
the vision projector automatically), drop in an image of one system
(a single line of music cropped to the staff — not a full page), and ask:
"Transcribe this sheet music to rokot-ABC."
A note on quantization: we deliberately do not publish 4-bit builds.
This model writes a strict notation language where a single wrong token
corrupts the score, and in our tests Q4 visibly degraded it. Q8_0 matched
the full-precision model exactly on our eval samples — use that.
Input images should be one system (one line of music) cropped to the
staff, ideally around 1400 px wide. The output is a complete, self-contained
ABC tune with fixed header order (X, M, L, K, …) — one deterministic
spelling for everything, so downstream tooling can rely on it.
Support the project ❤️
This model is free for personal and research use, and it exists because of
RokoTMIDI — a compact MIDI controller with the
button layout of a chromatic button accordion. If this model saves you hours
of transcription work and you'd like to give something back, the nicest way
to do it is to grab a controller (or tell an accordionist friend about it):
Every controller sold funds more GPU hours, better datasets, and future
versions of this model — including the handwritten-manuscript support that
is currently in training. Thank you! 🪗
License
Released under CC BY-NC 4.0 — free to use, share and build on for
non-commercial purposes with attribution. For commercial licensing,
get in touch via rokotmidi.com.
Limitations
One part / max two staves per system; complex piano textures with more
than two voices per staff are simplified.
Handwritten manuscripts are not yet supported (next training run).
Lyrics in non-Latin scripts remain less reliable than notation.
Very low-resolution or heavily skewed crops degrade accuracy — deskew and
crop per system for best results.
