Adapter: DPO Fine-Tuned Model (LLM Judge Dataset)

🧠 Base Model

• meta-llama/Llama-3.2-1B-Instruct
• Fine-tuned via TRL’s DPOTrainer using the LLM Judge preference dataset with 50 human-labeled comparisons.

⚙️ Training Configuration

📊 Dataset Overview

• File: llm_judge_preferences.csv
• Includes 50 preference examples with prompt, chosen, and rejected labels.

📁 Output Location

• Adapter uploaded to Hugging Face Hub at: sahithimuppavaram/dpo-llmjudge-lora-adapter