from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "nkuniyil/mistral-7b-sft")

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("nkuniyil/mistral-7b-sft")

# Generate
prompt = "### Instruction:\nWrite a haiku about AI.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

Part of SFT vs DPO Comparison Study

Base Model: mistralai/Mistral-7B-v0.1
Method: LoRA (r=16, alpha=32)
Dataset: Alpaca-GPT4 (10K examples)
Epochs: 3
Learning Rate: 2e-5
Training Time: ~6 hours on A100

This model is part of a study comparing Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO).