You can deploy this model and start earning money today!

HoangTran223/
SPPO_Qwen2.5-1.5B

Name: SPPO_Qwen2.5-1.5B
Availability: InStock

Views

No views yet

SPPO Qwen2.5-1.5B (UltraChat50k)

Self-Play Preference Optimization on Qwen2.5-1.5B: SFT on UltraChat200k, then 3 self-play iterations on UltraChat50k.

There is no iter0. Iteration 1 starts from the UltraChat200k SFT checkpoint (model_hub/Qwen2.5-1.5B-ultrachat200k / AIR-hl/Qwen2.5-1.5B-ultrachat200k).

Each iterN/ folder is a loadable causal LM plus that round's preference data and logs.

python

from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("HoangTran223/SPPO_Qwen2.5-1.5B", subfolder="iter3")

Path	Contents
`iterN/`	Final HF weights after 2 training epochs
`iterN/synthetic_data/`	Preference parquet (`train.parquet`, `test.parquet`) used by the trainer
`iterN/generated/`	Combined generations (`train.parquet`), PairRM `probabilities.json`, and raw `responses_*.json`
`iterN/ranking/`	PairRM rank matrix (`0_0.npy`)
`iterN/logs/`	vLLM generate log + PairRM rank log

Training: max_length=1024, max_prompt_length=512, batch 1 × grad accum 2, RMSProp, LR 5e-7.