from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load
base_model = AutoModelForCausalLM.from_pretrained("gpt2")
model = PeftModel.from_pretrained(base_model, "kkenna/kanye-west-gpt2")
tokenizer = AutoTokenizer.from_pretrained("kkenna/kanye-west-gpt2")

# Generate
prompt = "<|lyrics|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.8, do_sample=True)
print(tokenizer.decode(outputs[0]))

Training

Base: gpt2
Method: LoRA (r=16, alpha=32)
Epochs: 3
Learning Rate: 0.0002

Disclaimer

For educational/research purposes. Song lyrics are copyrighted.