This is a compact, 34-million-parameter causal language model for generating symbolic chord-progression continuations. It uses a Llama-style decoder architecture trained from scratch on tokenized chord sequences from Chordonomicon. It is not a fine-tune of Meta Llama weights.
Given a progression such as Cmaj7, Am7, Dm7, G7, the model samples possible continuations. Generation can be conditioned on a genre and time signature represented by control tokens.
This repository includes the weights, custom chord tokenizer, inference script, training script, preprocessing utilities, metrics, and training manifest needed to inspect and reproduce the work. Because the tokenizer represents structured chord events rather than natural-language text, the standard Hugging Face text-generation widget and AutoTokenizer are intentionally disabled.
Model details
Property
Value
Architecture
LlamaForCausalLM, trained from scratch
Parameters
34,008,576
Vocabulary
435 tokens
Context length
256 tokens
Hidden size
512
Intermediate size
2,048
Decoder layers
8
Attention heads
8
Saved weight dtype
FP32
Training precision
Mixed FP16
Transformers version
4.57.6
Weight size
about 130 MiB
The vocabulary contains 108 chord tokens, six special/event tokens, 12 genres, 179 subgenres, 128 styles, and two time-signature controls. The available top-level genres are:
alternative, country, electronic, jazz, metal, pop, pop-rock, punk, rap, reggae, rock, and soul.
The vocabulary contains 4/4 and 3/4 time-signature tokens. This training run's retained examples used 4/4; see the limitations below before treating 3/4 as a supported condition.
Live demo
Try the deployed CPU demo at Chord Progression Lab. The service scales to zero when idle, so the first page load can take roughly 20–40 seconds; generation is much faster after it wakes up.
The public demo is capped at one half-core CPU container, serializes model generations, limits its pending queue, and validates all generation controls server-side.
Quick start
Install the inference dependencies:
python -m pip install -r requirements.txt
Download the complete repository, including its custom tokenizer and helper code:
The command prints JSON containing the complete token sequence, decoded controls, and chord list for each sampled candidate. Candidate outputs are stochastic; changing the seed, temperature, or nucleus-sampling threshold changes the result.
Local web demo
The repository also contains the tested Gradio interface intended for the companion Hugging Face Space. After downloading the repository, launch it locally with:
The app uses the downloaded weights directly. It accepts comma-, bar-, semicolon-, or newline-separated chord symbols and exposes the supported genre and sampling controls.
Deploy a free-tier Modal copy
The exact serverless deployment configuration is included in modal_app.py. After installing and authenticating the Modal CLI, deploy from the downloaded repository root:
bash
1python -m pip install"modal>=1,<2"2modal token new
3modal deploy modal_app.py
The weights are bundled into the private Modal image during deployment. The running app does not receive a Hugging Face or Modal credential. See MODAL.md for the resource and scale-to-zero configuration.
Python usage
The included script shows the full supported path. At its core, loading uses the custom vocabulary alongside the standard Transformers model class:
Use ChordTokenizer.tokenize_progression(...) to construct the control and chord tokens before calling model.generate(...); sample_chord_lm.py is a complete example.
Training data and preprocessing
The model was trained on a processed subset of the Chordonomicon training split:
The preprocessing job inspected the first 25,000 source rows.
22,601 rows were retained and 2,399 were skipped.
2,370 rows were skipped because they contained unsupported chord symbols.
29 rows were skipped because they had fewer than four chord events.
The retained corpus averaged 81.5236 chord events and 167.2249 model tokens per row.
The longest sequence before truncation contained 3,673 tokens; training examples were truncated to 256 tokens.
A seeded random 95/5 row split produced 21,471 training examples and 1,130 evaluation examples.
Chord symbols were normalized into a deliberately bounded vocabulary. Rows containing chord qualities outside that vocabulary—including many extended, altered, slash, or otherwise uncommon forms—were dropped rather than mapped approximately. Metadata values were represented with genre, subgenre, style, and time-signature control tokens when available.
Training procedure
The saved top-level weights are the checkpoint with the best evaluation loss selected during training (checkpoint-2000). The run completed 10 epochs in about 41 minutes and 21 seconds. The exact GPU model, Python version, PyTorch version, and source-dataset revision were not recorded in the exported manifest.
These are token-level held-out metrics on a random row split of the processed corpus. They do not measure musical quality, originality, stylistic faithfulness, or listener preference. No external benchmark or listening study was performed.
Intended use
This model is intended for non-commercial research, education, prototyping, and exploration of symbolic chord generation. Appropriate uses include:
proposing chord continuations for a human to review;
studying compact models for structured musical sequences;
comparing sampling or conditioning strategies; and
reproducing or extending the training pipeline under the applicable license.
The outputs should be treated as suggestions, not authoritative music-theory judgments. Review generated material before using it in a composition or downstream system.
Limitations
The model generates symbolic chord labels only. It does not generate audio, MIDI, melody, voicing, performance, or reliable rhythm.
A <bar> token is a structural separator learned from preprocessing; it is not a guarantee that the preceding chords fill a metrically correct bar.
The source subset was the first 25,000 rows rather than a shuffled or comprehensively sampled slice.
The evaluation split was random by row, not grouped by artist or song. Possible near-duplicate or related-song leakage was not measured.
Although the vocabulary contains a 3/4 token, this run's retained training examples used 4/4; 3/4 behavior is therefore not established.
Unsupported and uncommon chord spellings were filtered out, narrowing the musical vocabulary and biasing the retained corpus.
Long source progressions were truncated to the 256-token context window.
Genre and style metadata are unevenly distributed and may be incomplete or noisy.
The model can repeat itself, end early, produce harmonically awkward sequences, or weakly follow requested metadata.
No memorization audit, originality analysis, listening study, demographic bias analysis, or external musical evaluation has been completed.
License and attribution
The model weights and repository materials are released under CC BY-NC 4.0 for non-commercial use with attribution.
This model is a transformation of data from Chordonomicon, which is distributed under CC BY-NC 4.0. The source data was filtered, normalized, tokenized, split, and used to train new model weights. See the Chordonomicon paper for the dataset methodology and requested citation.
The license does not eliminate possible third-party rights in individual musical works represented by the source data. Users are responsible for assessing their intended use, especially before redistributing outputs or using them beyond research and experimentation.
Repository contents
model.safetensors — trained model weights
config.json and generation_config.json — Transformers configuration
chord_tokenizer_vocab.json — complete custom vocabulary and control values
sample_chord_lm.py — command-line inference example
app.py and requirements-demo.txt — local Gradio interface
modal_app.py and MODAL.md — reproducible serverless demo deployment
backend/ — tokenizer, predictor, and harmony helpers
training/ — corpus utilities, training script, requirements, and research notebook
training_manifest.json and metrics.json — exported run metadata and metrics
Citation
If you use this model, cite this model repository and the underlying Chordonomicon dataset/paper. A formal model-specific citation will be added if an archival release is created.
