This dataset is used in the paper Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data.
Code:
https://github.com/mzhaoshuai/RefAlign
This dataset is modified from
https://huggingface.co/datasets/princeton-nlp/llama3-ultrafeedback. We use the BERTScore to choose the chosen and rejected responses.
Item with key ['Llama3.3-70B-Inst-Awq'] is the reference answers generated by… See the full description on the dataset page:
https://huggingface.co/datasets/mzhaoshuai/llama3-ultrafeedback-bertscore-bart-large-mnli.