Dataset Overview
This Dataset consists of the following open-sourced preference dataset
Arena Human Preference
Anthropic HH
MT-Bench Human Judgement
Ultra Feedback
Tulu3 Preference Dataset
Skywork-Reward-Preference-80K-v0.2
Cleaning
Cleaning Method 1: Only keep the following Language using FastText language detection(EN/DE/ES/ZH/IT/JA/FR)
Cleaning Method 2: Remove duplicates to ensure each prompt appears only once
Cleaning Method 3: Remove datasets where… See the full description on the dataset page: https://huggingface.co/datasets/yufan/Preference_Dataset_Merged.