Backups of the training runs for Nash Bargaining Preference Optimization (NBPO). They exist so a run can resume after the compute container is recycled; treat them as run state, not as a release.

Panels: helpsteer2 and hs2gemma are the four-judge HelpSteer2 panel with a Zephyr-7B-SFT and a Gemma-2-9B-it reference respectively; saferlhf is the two-judge safety panel. A directory named rmpool_mb<tag><r> is round r of the chain <tag>, whose anchor is round r-1 of the same chain.

Restore with resume_from_hf.sh from the project repository.

Each chain's most recent completed round, refreshed as the run advances. Older rounds are intentionally not kept: resuming a chain only needs its last policy.

Models under hs2gemma/ are derived from google/gemma-2-9b-it and are distributed under the Gemma Terms of Use; the Gemma Prohibited Use Policy applies to them and to any further derivative. Models under helpsteer2/ and saferlhf/ are derived from alignment-handbook/zephyr-7b-sft-full (Apache-2.0, itself derived from Mistral-7B).