A KTO signal transformed version of the highly loved Argilla DPO Mix, which is small cocktail combining DPO datasets built by Argilla with distilabel. The goal of this dataset is having a small, high-quality KTO dataset by filtering only highly rated chosen responses.
KTO matches or exceeds DPO performance at scales from 1B to 30B parameters.1 That is, taking a preference dataset of n… See the full description on the dataset page:
https://huggingface.co/datasets/argilla/kto-mix-15k.
