You can deploy this model and start earning money today!

HoangTran223/
S-SPPO_Qwen2.5-1.5B

Views

No views yet

S-SPPO Qwen2.5-1.5B (UltraChat50k)

Semantic-Calibrated Self-Play Preference Optimization (S-SPPO) on Qwen2.5-1.5B: SFT on UltraChat200k, then 3 self-play iterations on UltraChat50k.

There is no iter0. Iteration 1 starts from the UltraChat200k SFT checkpoint (model_hub/Qwen2.5-1.5B-ultrachat200k / AIR-hl/Qwen2.5-1.5B-ultrachat200k).

Each iterN/ folder is a loadable causal LM plus that round's preference data, generations, ranking, and logs.

python

from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("HoangTran223/S-SPPO_Qwen2.5-1.5", subfolder="iter3")

Path	Contents
`iterN/`	Final HF weights after 2 training epochs
`iterN/synthetic_data/`	Preference parquet (`train.parquet`, `test.parquet`) used by the trainer, including `sim_score_*`
`iterN/generated/`	Combined generations (`train.parquet`), PairRM `probabilities.json`, and raw `responses_*.json`
`iterN/ranking/`	PairRM rank matrix (`0_0.npy`)
`iterN/logs/`	vLLM generate log + PairRM rank log

Training: loss_type=sspppo, lam=1.0, beta=1e-3, max_length=1024, max_prompt_length=512, batch 1 × grad accum 2, RMSProp, LR 5e-7. Similarity scores from sentence-transformers/all-MiniLM-L6-v2.

HoangTran223/S-SPPO_Qwen2.5-1.5B

S-SPPO Qwen2.5-1.5B (UltraChat50k)

HoangTran223/
S-SPPO_Qwen2.5-1.5B