🎹 Change is Key: A Generative Framework for Controllable Musical Modulations
This repository contains the official implementation for "Change is Key: A Generative Framework for Controllable Musical Modulations".
The framework formulates musical modulation as a generative Fill-In-The-Middle (FIM) task using a LLaMA-based architecture. It allows users to select a structural turning point in a MIDI composition, transpose subsequent material to a target key, and autonomously or interactively synthesize a smooth, musically coherent transition bridge.
⚙️ Installation
First, clone this repository and navigate into the modulator folder:
Set up your environment using Conda:
module load cuda/12.4.0-gcc-12.2.0 # Optional: For HPC clusters
conda create -n modulate python=3.11 -y
conda activate modulate
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y
Install the project and its dependencies:
pip install -e .
🚀 Quickstart & Usage
The primary script for users is modulate_midi.py. It automatically fetches our pre-trained model weights directly from
Hugging Face (keshavbhandari/modulator).
Run the commands below from the root modulator folder.
1. Autonomous Infilling (No Chord Conditioning)
The model relies strictly on left/right musical context and temporal boundary tokens to autonomously bridge into the new key.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 4 \
--chord_mode none
2. Rule-Guided Modulation (Dynamic Chord Baseline)
Condition generation on a dynamically calculated functional chord progression that back-cycles diatonic chords to resolve smoothly into the target key.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 5 \
--chord_mode dynamic
3. LLM-Guided Modulation (Groq API)
Use a Large Language Model (via the Groq API) to plan custom harmonic progression instructions for the generative bridge based on the surrounding harmonic context.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 2 \
--chord_mode llm \
--api_key YOUR_GROQ_API_KEY
4. Optimal Modulation Boundary Detection
Automatically analyze the harmony to identify a structural turning point (e.g., resolving to the tonic) rather than cutting at a generic midpoint.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--enable_optimal_modulation_point \
--gap_duration 10000
(Note: --gap_duration is specified in milliseconds. 10000 = 10 seconds).
5. Manual Timestamp Control (Milliseconds)
Explicitly define exact start and end boundaries (in milliseconds) for the left context, the generative gap, and the right context.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--left_segment_start_time 70000 \
--left_segment_end_time 110000 \
--right_segment_start_time 117000 \
--right_segment_end_time 140000 \
--semitone_shift 7
6. Simple Infilling (Zero Key Change)
Repair or infill missing segments without transposing the target key by setting the semitone shift to 0.
python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 0 \
--chord_mode none
🎛️ Script Arguments Reference
| Argument | Default | Description |
|---|
| --midi_file | Required | Path to the input .mid file. |
| --output_path | infilled_output.mid | Target save location for output MIDI. |
| --semitone_shift | 4 | Semitone transposition for target key (0 for simple infilling). |
| --chord_mode | none | Conditioning strategy: none, dynamic, or llm. |
| --api_key | None | Groq API key (required when --chord_mode llm). |
| --enable_optimal_modulation_point | False | Analyzes harmony to pick the best structural cut point. |
| --left_segment_start_time | None | Manual left segment start (ms). |
| --left_segment_end_time | None | Manual left segment end / gap start (ms). |
| --right_segment_start_time | None | Manual gap end / right segment start (ms). |
| --right_segment_end_time | None | Manual right segment end (ms). |
| --left_segment_duration | 30000 | Fallback left context duration (ms). |
| --right_segment_duration | 30000 | Fallback right context duration (ms). |
| --gap_duration | 5000 | Fallback generated bridge duration (ms). |
| --temperature | 1.0 | Sampling temperature for the LLaMA model generation. |
🧠 Model Training
If you would like to continue training the model or train it on your own custom datasets, you can launch the training script using Accelerate:
accelerate launch --num_processes 1 /data/home/acw769/modulator/modulate/models/infiller_llama/train.py
📜 Citation
If you use this code, model weights, or evaluation framework in your research, please cite our paper:
@article{bhandari2026change,
title={Change is Key: A Generative Framework for Controllable Musical Modulations},
author={Bhandari, Keshav and Bizzarri, Matteo and Wiggins, Geraint A. and Colton, Simon},
year={2026}
}
