HelpSteer3 is an open-source dataset (CC-BY-4.0) that supports aligning models to become more helpful in responding to user prompts.
Preference Score Integer from -3 to 3, corresponding to:
-3: Response 1 is much better than Response 2
-2: Response 1 is better than Response 2
-1: Response 1 is slightly better than Response 2
0: Response 1 is about the same as Response 2
1: Response 2 is slightly better than Response 1
2: Response 2 is better than Response… See the full description on the dataset page:
https://huggingface.co/datasets/CarrotAI/HelpSteer3_dpo_format.
