neovalle/H4rmony_dpo in ChatML format, ready to use in HuggingFace TRL's DPO Trainer.
Python code used for conversion:
from datasets import load_dataset
def format(columns):
return {
"prompt": f"<|im_start|>user\n{columns['prompt']}<|im_end|>\n<|im_start|>assistant\n",
"chosen": f"{columns['chosen']}<|im_end|>",
"rejected": f"{columns['rejected']}<|im_end|>",
}… See the full description on the dataset page:
https://huggingface.co/datasets/Felladrin/ChatML-H4rmony_dpo.
