You can deploy this model and start earning money today!

3podi/
charter-v1.0-40-M-best-acc-nonpad

Name: charter-v1.0-40-M-best-acc-nonpad
Availability: InStock

pytorch

2511.03337

Views

No views yet

audio2chart - charter v1.0

charter-v1.0 is a pretrained Transformer-based autoregressive model that generates Guitar Hero style .chart files directly from audio.
It is part of the audio2chart framework for end-to-end audio-to-chart generation, and its full methodology and architecture are described in our arXiv publication.

Model Overview

The model follows a multimodal encoder–decoder Transformer design:

Audio Encoder: frozen pretrained Encodec model converting raw waveform into discrete audio codes.
Adapter: lightweight 1D convolution aligns audio embeddings to token time resolution.
Decoder: autoregressive Transformer with cross-attention to encoded audio, trained to predict discrete chart tokens (notes/chords) at fixed 20 ms or 40 ms intervals.
Tokenization: 63 categorical tokens representing all valid fret-button combinations plus padding for silent steps.
Training Data: ~10 k paired audio–chart samples.
Model Sizes: 25 M (S) and 227 M (M) parameters.

The 40 ms medium model (3podi/charter-v1.0-40-M-best-acc-nonpad) provides the best quality–speed trade‑off.

Quick Usage

🔹 Local Installation

bash

git clone https://github.com/3podi/audio2chart.git
cd audio2chart
pip install -r requirements.txt

🔹 Load the model from Hugging Face

python

from inference.engine import Charter

# Load the pretrained Charter model
model = Charter.from_pretrained("3podi/charter-v1.0-40-M-best-acc-nonpad")

# Generate chart tokens from an audio file
seqs = model.generate("path/to/song.mp3")

🔹 Or use the CLI script

python generate.py path/to/song.mp3 --model_name 3podi/charter-v1.0-40-M-best-acc-nonpad --output my_songs/