DPO Fine-Tuned Adapter - LLM Judge Dataset

🧠 Model

• Base: meta-llama/Llama-3.2-1B-Instruct
• Fine-tuned using TRL's DPOTrainer with the LLM Judge preference dataset (50 pairs)

⚙️ Training Parameters

📦 Dataset

• Source: llm_judge_preferences.csv
• Size: 50 human-labeled pairs with prompt, chosen, and rejected columns

📂 Output

• Adapter saved and uploaded as Likhith003/dpo-llmjudge-lora-adapter