Language Models such as GPT-2 can be used for Music Generation. The idea is to represent pieces of music as texts, effectively reducing the task to Language Generation.
This model is a rather small instance of GPT-2 trained on
TristanBehrens/js-fakes-4bars. The model generates 4 bars at a time of Bach-like chorales with four voices (soprano, alto, tenor, bass).
If you are contribute, if you want to say hello, if you want to know more, find me on
LinkedInThe model is GPT-2 with 6 decoders and 8 attention-heads each. The context length is 512. The embedding dimensions are 512 as well. The vocabulary size is 119.
This model is just a proof of concept. It shows that HuggingFace can be used to compose music.
There is a notebook in the repo that you can run on Google Colab.
Since this model has been trained on a very small corpus of music, it is overfitting heavily.
