A binarized version of the UltraSafety dataset by openbmb which is ready to be used with DPO, ORPO, etc.
Extra filtering is advised.
Created new rating as the average of the helpfulness and honesty ratings (if in numeric format)
Kept highest-rated response as "chosen response" and lowest-rated response as "rejected response"
System and user messages were extracted from custom system prompts of each candidate response
