The SMT model fine-tuned on the
Camera GrandStaff dataset for pianoform transcription.
The code of the model is hosted in
this repository.
The SMT model consists of a vision encoder (ConvNext) and a text decoder (classic Transformer).
Given an image of a music system, the encoder first encodes the image into a tensor of embeddings (of shape batch_size, seq_len, hidden_size), after which the decoder autoregressively generates text, conditioned on the encoding of the encoder.
1@misc{RiosVila2024,
2 title={Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription},
3 author={Antonio Ríos-Vila and Jorge Calvo-Zaragoza and Thierry Paquet},
4 year={2024},
5 eprint={2402.07596},
6 archivePrefix={arXiv},
7 primaryClass={cs.CV},
8 url={https://arxiv.org/abs/2402.07596},
9}
