You can deploy this model and start earning money today!

Sudhanshu1985/
slm-preference-pairs

Views

No views yet

slm preference pairs (DPO / RLAIF)

AI-feedback preference pairs used to align the legal/financial SLMs via DPO and RLAIF. On-policy candidates were sampled from each SFT model and ranked by gpt-4.1-mini into (chosen, rejected) pairs.

File Model Pairs

125m_pairs.jsonl slm-125m-sft 279

500m_pairs.jsonl thesreedath/slm-500m-qa 163

gemma_pairs.jsonl gemma-2-2b-legal-sft 189

Schema: 125M/500M = {system, user, chosen, rejected, chosen_score, rejected_score}; Gemma… See the full description on the dataset page: https://huggingface.co/datasets/Sudhanshu1985/slm-preference-pairs.