Aegis-Safety-DPO is a high-density, (mostly) manually-curated preference dataset designed for Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO).
Unlike traditional safety datasets that train models to be "preachy," "evasive," or "apologetic", Aegis trains models that refuse to answer