import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

tokenizer  = AutoTokenizer.from_pretrained("Chandrudp29/gpt2-lora-aisafety")
base_model = AutoModelForCausalLM.from_pretrained("gpt2")
model      = PeftModel.from_pretrained(base_model, "Chandrudp29/gpt2-lora-aisafety")
model.eval()

prompt = "AI alignment is the challenge of"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=80,
        temperature=0.7,
        do_sample=True,
        top_k=50,
        repetition_penalty=1.2,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details


Base model	`gpt2` (117M params)
Adapter type	LoRA
LoRA rank	r=8
lora_alpha	16
target_modules	c_attn, c_proj
Trainable params	811,008 (0.65%)
Training examples	30 AI safety texts
Epochs	20
Learning rate	2e-4
Hardware	NVIDIA T4 (16GB)
Training time	18 seconds
Adapter size	3.25 MB

Training Topics

The model was fine-tuned on short texts covering:

AI alignment and value alignment
Mechanistic interpretability and circuits
Sparse autoencoders and superposition
Constitutional AI and RLHF
Reward hacking and goal misgeneralization
Corrigibility and AI governance
Scalable oversight and debate

Example Prompts

AI alignment is the challenge of...
Mechanistic interpretability seeks to...
Constitutional AI is a method for...
Corrigibility refers to an AI system that...
The risks of advanced AI include...

Code

Training code: chandrudp29/lora-finetune

About LoRA

LoRA freezes pre-trained model weights and injects trainable low-rank matrices into each transformer layer. Instead of training all 125M parameters, only the two small adapter matrices (A × B where rank=8) are updated — reducing trainable parameters by 99.35%.

See: LoRA paper (Hu et al., 2021)