This dataset was curated specifically for training the wzhouad/zephyr-7B-WPO-HB model in a hybrid RL setting. The prompts are sourced from the Ultrafeedback dataset, and the corresponding outputs are as follows:
On-Policy Outputs: 5 outputs generated using the HuggingFaceH4/mistral-7b-sft-beta model, based on Ultrafeedback prompts.
GPT-4-turbo Outputs: 1 output generated using GPT-4-turbo, based on the same Ultrafeedback prompts.
