Adapter: DPO Fine-Tuned Model (PairRM Dataset)

🧠 Base Model

• meta-llama/Llama-3.2-1B-Instruct
• Fine-tuned using the DPOTrainer from TRL on the PairRM preference dataset consisting of 500 preference pairs.

⚙️ Training Configuration

📊 Dataset Overview

• File: pairrm_preferences.csv
• Contains 500 entries with fields: prompt, chosen, and rejected.

📁 Output Location

• Adapter uploaded to Hugging Face Hub at: sahithimuppavaram/dpo-pairrm-lora-adapter