RLAIF (GRPO (group-relative RLHF, no critic) with an AI-feedback reward model) refinement of sudhisrk1982/slm-125m-instruct, trained on 500 AI-generated preference triplets (chosen from Gemini 2.5 Flash-Lite, rejected sampled from the SFT model itself, judged by Gemini 2.5 Flash).

Result: OVER-OPTIMIZED / DEGRADED. Do not use for inference. GRPO learned well for ~35 steps (reward -0.28 -> -0.04, KL ~15) but in the final steps the KL to the reference ran away to ~93 and the policy collapsed into degenerate, non-coherent text. Only the final checkpoint is saved, so THESE published weights are the collapsed ones. Published as a teaching artifact of KL over-optimization -- the classic RLHF failure mode -- NOT as a usable model. A corrected run would checkpoint the best-by-reward step and stop earlier.

Not legal advice.