The wise-data-preferences dataset is a synthetically created collection of values-laden conversations with preferred and rejected responses, designed to train language models to provide more nuanced and helpful responses to harmful, heavy, or exploratory questions. This dataset was specifically created to train the WiseLLama-8B model, a LLaMa-3.1-8B-Instruct model fine-tuned using DPO (Direct Preference Optimization).
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/meaningalignment/wise-data-preferences.
