This repository contains a preference dataset designed for Direct Preference Optimization (DPO) training. The preferences were generated using a custom-built LLM Judge powered by GPT-3.5-Turbo.
This dataset was created as part of the "Preference Dataset Collection and DPO Training" project to fine-tune meta-llama/Llama-3.2-1B-Instruct.
Fine-tuned Model: NilayR/llama32-dpo-llm-judge
Dataset Details
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/NilayR/llm-judge-preferences-llama32.
