You can deploy this model and start earning money today!

eoinedge/
irish-abc-qwen2-5-lora

Name: irish-abc-qwen2-5-lora
Availability: InStock

Qwen/Qwen2.5-0.5B-Instruct

Views

No views yet

Model Card for eoinedge/irish-abc-qwen2-5-lora

This adapter is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct. It was trained by Eoin Jordan to improve ABC notation continuation for Irish traditional-style music generation.

Model description

eoinedge/irish-abc-qwen2-5-lora is a LoRA adapter that steers the base Qwen2.5 instruction model toward generating valid ABC music notation in an Irish traditional style. The adapter was trained using parameter-efficient fine-tuning with PEFT and the Hugging Face TRL SFT framework.

Usage

Load the base model and adapter together to generate ABC continuations:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "eoinedge/irish-abc-qwen2-5-lora",
    trust_remote_code=True,
    use_fast=True,
)
model = PeftModel.from_pretrained(base_model, "eoinedge/irish-abc-qwen2-5-lora")

prompt = "K:D\n|:A2FA DAFA|"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=180)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training data

The adapter was trained on a curated ABC continuation dataset stored in this repository:

Training set: data/processed/train_10k.jsonl
Validation set: data/processed/val_1k.jsonl

The dataset is derived from public-domain and traditional ABC sources, formatted for supervised continuation training.

Training setup

Base model: Qwen/Qwen2.5-0.5B-Instruct
Adapter type: LoRA (peft)
Sequence length: 512 tokens
Epochs: 1
Learning rate: 2e-4
Batch size: 1
Gradient accumulation steps: 16
LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Gradient checkpointing: enabled
Learning rate scheduler: cosine
Warmup steps: 100
Precision: fp16
Trainer: trl.SFTTrainer

The training configuration is defined in configs/qwen2_5_0_5b_lora_mac.yaml and the fine-tuning script is scripts/train_lora.py.

Evaluation

Local evaluation was run on 17 May 2026 using the repository's generation tooling and the adapter artifact in outputs/adapters/irish-abc-qwen2_5-0_5b-lora-mac.
Deterministic inference (do_sample=False) with the current local MPS environment produced continuations dominated by repeated punctuation characters, indicating unstable adapter generation behavior for the tested ABC prompt.
Sampled generation with the current local setup also encountered numerical instability during prediction, raising an invalid logits error (probability tensor contains either inf, nan or element < 0).
These results are preliminary; further prompt tuning, adapter refinement, or a different inference runtime is recommended before relying on this adapter for production ABC generation.

Limitations

This is an adapter, not a standalone full model; it requires the base Qwen model.
Generated ABC should be reviewed for musical correctness and notation validity.
The adapter may still produce invalid measures or non-ABC text in some cases.
Do not use this model to generate copyrighted music without permission.

Citation

If you use this adapter, cite the adapter itself:

bibtex

@software{jordan2026irishabcqwen,
  author = {Jordan, Eoin},
  title = {Irish ABC Qwen2.5 LoRA Adapter},
  year = {2026},
  url = {https://huggingface.co/eoinedge/irish-abc-qwen2-5-lora}
}

The adapter uses the TRL and PEFT libraries. Use their current project citation files when citing those dependencies.