A fine-tune of NotaGen-medium on ~125 traditional Chinese guzheng pieces in interleaved ABC notation. NotaGen is a hierarchical patch+character GPT pre-trained on ~1M Western scores; this fine- tune adapts it to generate single-voice guzheng music in pentatonic scales.

Model

Spec	Value
Architecture	Patch-level GPT (16 layers) + Char-level GPT (3 layers)
Hidden size	1024
Total parameters	~231M
Patch size	16 chars
Max patches per sequence	1024
Vocab	128 (ASCII)
Base model	`ElectricAlexis/NotaGen` (medium pretrained)

Training

Data: 125 pieces (26 hand-curated guzheng repertoire + 99 Guzheng Tech99), augmented to all 15 keys via abctoolkit transposition. Bundled in data/dataset.jsonl of the GitHub repo.
Sampling: triangular ±3-semitone key augmentation per epoch + 5× repertoire oversampling via WeightedRandomSampler.
Optimizer: AdamW, constant LR 1e-5 with 100-step warmup.
Hardware: Apple MPS, fp32, batch size 1.
Best epoch: 9, eval loss 0.471.

Usage

python

import torch
from safetensors.torch import load_file
from transformers import GPT2Config
# (Add NotaGen's `finetune/` to PYTHONPATH; see github.com/anjieliu/guzheng)
from utils import Patchilizer, NotaGenLMHeadModel

state = load_file("model.safetensors")
patch_cfg = GPT2Config(num_hidden_layers=16, max_length=1024,
                       max_position_embeddings=1024, n_embd=1024,
                       num_attention_heads=16, vocab_size=1)
char_cfg = GPT2Config(num_hidden_layers=3, max_length=17,
                      max_position_embeddings=17, hidden_size=1024,
                      num_attention_heads=16, vocab_size=128)
model = NotaGenLMHeadModel(encoder_config=patch_cfg, decoder_config=char_cfg)
model.load_state_dict(state)
model.eval()

Full training/inference code lives in the GitHub repo: github.com/anjieliu/guzheng (notagen/generate.py is a working entry point).

Limitations

Trained on ~125 unique pieces — small corpus relative to standard generative-music datasets.
Generations are anchored to K:C M:4/4 L:1/32 metadata; the model has not seen other meters or unit notes.
Pentatonic / guzheng-idiomatic playing techniques (vibrato, glissando, keyswitches) are not encoded in the training data and so are not generated.
Outputs require the post-processing pipeline described in the thesis (D-centering + keyswitch insertion) to play correctly through guzheng sample libraries.

Citation

If you use this model, please cite the upstream NotaGen paper and the thesis (see github.com/anjieliu/guzheng).

License

Code: MIT (see GitHub repo). Weights and dataset: CC-BY-SA 4.0.

anjieliu/notagen-guzheng

NotaGen-medium fine-tuned for Guzheng