A model for generating symbolic drum patterns with a stacked LSTM +
multi-head attention model. The pipeline converts drum MIDI files into
integer token sequences, trains a next-token prediction model, and
autoregressively generates new drum MIDI (and optionally .wav) output.
Before training, raw MIDI files must be filtered to keep only drum tracks. This can be done in two ways:
EDA.ipynb – The exploratory data analysis notebook includes preprocessing steps that filter and inspect the drum data interactively. Clened files are saved in drums_only/.
lstm/ext_drums.py – A standalone script that downloads the Lakh MIDI Clean dataset from Kaggle and extracts drum-only MIDI files into drums_only/.
Training (Train.ipynb)
Reads MIDI files from drums_only/.
Keeps only drum-track note events during tokenisation.
Preprocesses up to 2 048 MIDI files into fixed-length token sequences.
Trains lstm models with the combined cross-entropy + KL + cosine-diversity
loss, cosine-decay learning rate, gradient clipping, and early stopping.
Saves the best checkpoint to models/best_<model_name>_<timestamp>.keras.
Key hyper-parameters (set at the top of the notebook):
Variable
Default
Description
TIME_STEP
0.1 s
Duration of one time-shift step
MAX_TIME_SHIFT
100
Largest time-shift token (→ 10 s)
SEQ_LEN
512
Context window in tokens
batch_size
8
Training batch size
epochs
100
Maximum training epochs
Inference (Inference.ipynb)
Loads a saved checkpoint and autoregressively samples a token sequence, then
converts it to drum MIDI and optionally WAV.
Key generation parameters (set at the top of the notebook):
Variable
Default
Description
TARGET_DURATION
20 s
Desired length of generated music
TEMPERATURE
0.9
Overall sampling temperature
TIME_SHIFT_TEMP
0.8
Separate temperature for timing tokens
TOP_K
32
Top-k filtering per step
TOP_P
0.95
Nucleus (top-p) sampling threshold
REPETITION_PENALTY
1.7
Down-weight recently used drum-hit tokens
WARMUP_TOKENS
512
Context warm-up tokens excluded from output
Output files are written to output/generated_<timestamp>.mid (and .wav).
Generated MIDI is exported as a drum instrument track.
Data Split (Train / Validation / Test)
To prevent data leakage and ensure rigorous evaluation, the MIDI corpus is split into three disjoint sets at the file level:
Set
Files
Purpose
Train
60% (~8678 files)
Training data for model learning
Validation
15% (~2170 files)
Tuning hyperparameters and early stopping
Test
25% (~3616 files)
Held-out evaluation only
Creating the split
Run once to generate the split files:
python lstm/split_data.py
This creates three JSON files inside the drums_only/ directory:
drums_only/train_files.json — File paths for training
drums_only/val_files.json — File paths for validation
drums_only/test_files.json — File paths for evaluation
The split uses a fixed random seed (SEED = 42) for reproducibility.
Using the split in Training
Train.ipynb automatically loads drums_only/train_files.json and drums_only/val_files.json in the "Data preparation" cell.
Training data only comes from the train set, validation only from the val set. Test files are never touched.
Using the split in Evaluation
Evaluation.ipynb loads drums_only/test_files.json to ensure generated samples are evaluated against held-out files.
This guarantees no test-set contamination in the evaluation metrics.
