This dataset takes the prompts of Magpie-Align/Magpie-Air-DPO-100K-v0.1 and samples responses using google/gemma-2-27b-it.
In particular, we sample 5 responses using vLLM with the following sampling configuration:
{n=5, temperature=0.9,top_p=1,max_tokens=4096, seed=42}
This configuration did not yield any samples with 5 same generated responses.
We then choose the best and worst samples as chosen/rejected responses using the Skywork/Skywork-Reward-Gemma-2-27B-v0.2 reward model.
Please cite… See the full description on the dataset page:
https://huggingface.co/datasets/yjwon/Magpie-Air-Gemma2-DPO-100K.
