This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_Soft_DPO_Qwen2.5MathRM72B_th0.5_pair4NoGT dataset. It achieves the following results on the evaluation set:

Loss: 1.1113
Rewards/chosen: -1.1800
Rewards/rejected: -6.6540
Rewards/accuracies: 0.7622
Rewards/margins: 5.4741
Logps/rejected: -53.0647
Logps/chosen: -38.2739
Logits/rejected: -2.5328
Logits/chosen: -2.7308

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 8.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.8097	0.0287	20	0.7468	-0.0650	-0.0192	0.4695	-0.0457	-47.5357	-37.3447	-2.1604	-2.3001
0.7098	0.0573	40	0.7392	-0.0260	-0.0092	0.4817	-0.0168	-47.5274	-37.3123	-2.1641	-2.3047
0.667	0.0860	60	0.7421	-0.0474	0.0085	0.4695	-0.0559	-47.5126	-37.3301	-2.1631	-2.3031
0.7548	0.1146	80	0.7376	0.0305	0.0255	0.4817	0.0050	-47.4985	-37.2652	-2.1631	-2.3039
0.7552	0.1433	100	0.7355	-0.0184	0.0558	0.4878	-0.0742	-47.4732	-37.3059	-2.1626	-2.3029
0.7159	0.1719	120	0.7366	-0.0684	-0.0071	0.4451	-0.0613	-47.5256	-37.3476	-2.1611	-2.3012
0.6866	0.2006	140	0.7321	-0.0317	-0.0018	0.4817	-0.0299	-47.5212	-37.3170	-2.1640	-2.3038
0.7233	0.2292	160	0.7263	-0.0373	-0.0365	0.5244	-0.0008	-47.5501	-37.3216	-2.1595	-2.2999
0.7034	0.2579	180	0.7341	0.0165	0.0212	0.4695	-0.0047	-47.5020	-37.2768	-2.1643	-2.3054
0.7143	0.2865	200	0.7105	0.0210	-0.0044	0.5244	0.0254	-47.5233	-37.2731	-2.1641	-2.3052
0.6639	0.3152	220	0.7046	-0.0765	-0.0831	0.4817	0.0066	-47.5889	-37.3543	-2.1645	-2.3061
0.626	0.3438	240	0.6985	-0.0902	-0.1901	0.6037	0.0999	-47.6781	-37.3657	-2.1536	-2.2951
0.6912	0.3725	260	0.6865	-0.1718	-0.2607	0.5427	0.0889	-47.7369	-37.4337	-2.1545	-2.2965
0.7198	0.4011	280	0.6858	-0.1616	-0.3038	0.5976	0.1422	-47.7728	-37.4252	-2.1561	-2.2989
0.6855	0.4298	300	0.6707	-0.1256	-0.3045	0.6098	0.1789	-47.7735	-37.3953	-2.1573	-2.3002
0.627	0.4585	320	0.6646	-0.2207	-0.3814	0.5671	0.1608	-47.8376	-37.4745	-2.1555	-2.2991
0.6619	0.4871	340	0.6684	-0.2056	-0.4176	0.5549	0.2120	-47.8677	-37.4619	-2.1556	-2.3006
0.6881	0.5158	360	0.6530	-0.1221	-0.3678	0.5732	0.2457	-47.8262	-37.3924	-2.1642	-2.3097
0.6702	0.5444	380	0.6378	-0.0841	-0.3701	0.6524	0.2860	-47.8281	-37.3607	-2.1667	-2.3130
0.7814	0.5731	400	0.6356	0.0574	-0.2797	0.6646	0.3371	-47.7528	-37.2428	-2.1710	-2.3176
0.5632	0.6017	420	0.6212	0.0633	-0.2793	0.6220	0.3426	-47.7524	-37.2378	-2.1753	-2.3226
0.6092	0.6304	440	0.6236	0.1639	-0.2869	0.7012	0.4509	-47.7588	-37.1539	-2.1769	-2.3246
0.5792	0.6590	460	0.6180	0.2082	-0.2888	0.7134	0.4969	-47.7603	-37.1171	-2.1811	-2.3307
0.7476	0.6877	480	0.6127	0.2526	-0.2462	0.7134	0.4988	-47.7248	-37.0801	-2.1811	-2.3306
0.6404	0.7163	500	0.6097	0.1475	-0.4108	0.6829	0.5583	-47.8620	-37.1676	-2.1753	-2.3255
0.6217	0.7450	520	0.6060	0.2524	-0.2667	0.6829	0.5191	-47.7419	-37.0802	-2.1832	-2.3343
0.5451	0.7736	540	0.6083	0.2531	-0.2904	0.6707	0.5436	-47.7617	-37.0796	-2.1757	-2.3258
0.6646	0.8023	560	0.5984	0.0430	-0.5552	0.6829	0.5982	-47.9823	-37.2548	-2.1690	-2.3204
0.7004	0.8309	580	0.6040	-0.0139	-0.7923	0.7195	0.7784	-48.1800	-37.3022	-2.1618	-2.3150
0.5622	0.8596	600	0.6062	0.0082	-0.7177	0.6768	0.7259	-48.1178	-37.2837	-2.1646	-2.3188
0.6623	0.8883	620	0.5950	0.1405	-0.5797	0.7012	0.7202	-48.0028	-37.1735	-2.1779	-2.3332
0.633	0.9169	640	0.5962	0.2780	-0.4278	0.6951	0.7058	-47.8762	-37.0589	-2.1870	-2.3420
0.6525	0.9456	660	0.5977	0.2796	-0.5464	0.6890	0.8260	-47.9751	-37.0576	-2.1819	-2.3382
0.606	0.9742	680	0.6027	0.1874	-0.6347	0.6951	0.8221	-48.0486	-37.1344	-2.1798	-2.3361
0.4513	1.0029	700	0.5964	0.0997	-0.7349	0.6951	0.8345	-48.1321	-37.2075	-2.1694	-2.3256
0.4754	1.0315	720	0.5880	0.2729	-0.6291	0.7317	0.9020	-48.0439	-37.0632	-2.1792	-2.3371
0.2628	1.0602	740	0.5852	0.2416	-0.6292	0.7073	0.8709	-48.0441	-37.0892	-2.1738	-2.3307
0.2967	1.0888	760	0.5790	0.4952	-0.4594	0.7195	0.9545	-47.9025	-36.8779	-2.1962	-2.3548
0.6242	1.1175	780	0.5842	0.3462	-0.7290	0.7012	1.0752	-48.1272	-37.0021	-2.1907	-2.3491
0.481	1.1461	800	0.5883	0.2286	-0.7513	0.7134	0.9799	-48.1458	-37.1001	-2.1808	-2.3387
0.61	1.1748	820	0.5986	0.1765	-0.7710	0.6585	0.9475	-48.1622	-37.1435	-2.1811	-2.3399
0.4287	1.2034	840	0.5885	0.0708	-0.9471	0.7012	1.0180	-48.3090	-37.2316	-2.1741	-2.3339
0.3261	1.2321	860	0.5897	0.1286	-0.8837	0.7317	1.0123	-48.2561	-37.1834	-2.1805	-2.3407
0.4974	1.2607	880	0.5993	0.1065	-0.8439	0.6707	0.9504	-48.2229	-37.2018	-2.1851	-2.3462
0.2233	1.2894	900	0.5777	0.2207	-0.8448	0.6951	1.0654	-48.2237	-37.1067	-2.1927	-2.3533
0.1847	1.3181	920	0.5935	0.2330	-0.8250	0.6707	1.0580	-48.2072	-37.0964	-2.2015	-2.3631
0.361	1.3467	940	0.5968	0.2147	-0.9912	0.6890	1.2059	-48.3457	-37.1117	-2.1924	-2.3518
0.4091	1.3754	960	0.5954	0.3821	-0.7541	0.7256	1.1362	-48.1481	-36.9721	-2.2101	-2.3711
0.223	1.4040	980	0.5829	0.2720	-0.9411	0.7012	1.2132	-48.3040	-37.0639	-2.2015	-2.3629
0.3438	1.4327	1000	0.5927	0.2181	-1.0220	0.6768	1.2401	-48.3714	-37.1088	-2.2091	-2.3736
0.3168	1.4613	1020	0.6046	0.1324	-1.0940	0.6951	1.2264	-48.4314	-37.1803	-2.2071	-2.3714
0.3262	1.4900	1040	0.5849	0.3006	-0.9467	0.6890	1.2474	-48.3086	-37.0400	-2.2146	-2.3776
0.3149	1.5186	1060	0.5896	0.3592	-0.8600	0.7134	1.2192	-48.2364	-36.9912	-2.2179	-2.3811
0.4568	1.5473	1080	0.5923	0.3145	-0.9415	0.7134	1.2560	-48.3043	-37.0285	-2.2184	-2.3821
0.4012	1.5759	1100	0.5937	0.1549	-1.0538	0.6707	1.2087	-48.3978	-37.1615	-2.2083	-2.3713
0.3457	1.6046	1120	0.5870	0.2303	-1.0264	0.7378	1.2567	-48.3750	-37.0987	-2.2130	-2.3767
0.2941	1.6332	1140	0.6070	0.0448	-1.2637	0.7073	1.3085	-48.5728	-37.2533	-2.2081	-2.3731
0.2909	1.6619	1160	0.6101	0.1544	-1.0593	0.7073	1.2137	-48.4024	-37.1619	-2.2121	-2.3760
0.4656	1.6905	1180	0.5988	0.2853	-0.9786	0.7012	1.2639	-48.3352	-37.0528	-2.2157	-2.3788
0.3255	1.7192	1200	0.5988	-0.0352	-1.2611	0.7195	1.2259	-48.5706	-37.3199	-2.2036	-2.3674
0.4172	1.7479	1220	0.6166	0.1941	-1.1268	0.7439	1.3209	-48.4587	-37.1288	-2.2206	-2.3835
0.2205	1.7765	1240	0.5990	0.0837	-1.2028	0.7012	1.2865	-48.5220	-37.2208	-2.2187	-2.3823
0.406	1.8052	1260	0.5968	0.0159	-1.4036	0.7317	1.4195	-48.6894	-37.2773	-2.2055	-2.3691
0.2753	1.8338	1280	0.5964	-0.0747	-1.4471	0.7378	1.3724	-48.7256	-37.3528	-2.2009	-2.3647
0.3774	1.8625	1300	0.6036	-0.3612	-1.7385	0.7195	1.3773	-48.9685	-37.5916	-2.1837	-2.3486
0.4699	1.8911	1320	0.6099	0.0843	-1.3682	0.7439	1.4524	-48.6598	-37.2204	-2.2067	-2.3706
0.2721	1.9198	1340	0.6013	0.0004	-1.3165	0.7134	1.3170	-48.6168	-37.2902	-2.2037	-2.3671
0.5676	1.9484	1360	0.5971	-0.1834	-1.5204	0.7012	1.3370	-48.7867	-37.4434	-2.1975	-2.3611
0.5476	1.9771	1380	0.5904	0.0251	-1.3444	0.7317	1.3694	-48.6400	-37.2697	-2.2057	-2.3693
0.1795	2.0057	1400	0.5867	0.1391	-1.2885	0.7622	1.4276	-48.5935	-37.1747	-2.2090	-2.3722
0.0391	2.0344	1420	0.6082	-0.2109	-1.6421	0.7256	1.4312	-48.8881	-37.4663	-2.1973	-2.3625
0.0631	2.0630	1440	0.6048	-0.2160	-1.7871	0.7317	1.5711	-49.0090	-37.4706	-2.2020	-2.3683
0.0467	2.0917	1460	0.5936	0.2394	-1.4106	0.7561	1.6501	-48.6952	-37.0911	-2.2386	-2.4063
0.0764	2.1203	1480	0.6123	0.2622	-1.5212	0.7439	1.7835	-48.7874	-37.0720	-2.2378	-2.4049
0.1373	2.1490	1500	0.6347	0.3097	-1.5617	0.7378	1.8714	-48.8211	-37.0325	-2.2556	-2.4238
0.0496	2.1777	1520	0.6155	0.3161	-1.5939	0.7744	1.9100	-48.8480	-37.0272	-2.2496	-2.4177
0.066	2.2063	1540	0.6185	0.0108	-1.8621	0.7561	1.8729	-49.0715	-37.2816	-2.2355	-2.4052
0.0661	2.2350	1560	0.6373	0.0213	-1.8370	0.7317	1.8583	-49.0505	-37.2728	-2.2452	-2.4154
0.1682	2.2636	1580	0.6381	0.1656	-1.7342	0.75	1.8998	-48.9649	-37.1526	-2.2525	-2.4234
0.1508	2.2923	1600	0.6338	0.3530	-1.5511	0.7378	1.9041	-48.8123	-36.9964	-2.2687	-2.4398
0.1198	2.3209	1620	0.6320	0.5016	-1.5296	0.7683	2.0312	-48.7944	-36.8726	-2.2754	-2.4455
0.0402	2.3496	1640	0.6491	0.4967	-1.4456	0.7622	1.9423	-48.7244	-36.8767	-2.2816	-2.4524
0.1509	2.3782	1660	0.6508	0.4741	-1.5187	0.7622	1.9928	-48.7853	-36.8955	-2.2806	-2.4520
0.2357	2.4069	1680	0.6597	0.4079	-1.6499	0.7561	2.0578	-48.8946	-36.9507	-2.2788	-2.4513
0.0964	2.4355	1700	0.6634	0.1205	-1.9334	0.7439	2.0539	-49.1309	-37.1902	-2.2611	-2.4330
0.0981	2.4642	1720	0.6613	-0.0037	-1.9793	0.7378	1.9756	-49.1691	-37.2937	-2.2542	-2.4270
0.0413	2.4928	1740	0.6528	-0.2074	-2.2347	0.7378	2.0273	-49.3820	-37.4634	-2.2421	-2.4157
0.1294	2.5215	1760	0.6528	-0.2629	-2.3152	0.7622	2.0524	-49.4491	-37.5096	-2.2408	-2.4132
0.102	2.5501	1780	0.6508	-0.3086	-2.3653	0.7561	2.0567	-49.4908	-37.5477	-2.2337	-2.4063
0.0579	2.5788	1800	0.6535	0.1429	-1.9645	0.7439	2.1074	-49.1568	-37.1714	-2.2710	-2.4450
0.111	2.6074	1820	0.6760	0.1204	-2.0361	0.7561	2.1566	-49.2165	-37.1902	-2.2767	-2.4511
0.0145	2.6361	1840	0.6705	0.0830	-2.0359	0.75	2.1189	-49.2163	-37.2214	-2.2617	-2.4367
0.222	2.6648	1860	0.6698	-0.1420	-2.2164	0.7134	2.0744	-49.3667	-37.4089	-2.2369	-2.4117
0.0942	2.6934	1880	0.6784	-0.0109	-2.1041	0.7439	2.0932	-49.2731	-37.2996	-2.2671	-2.4435
0.0935	2.7221	1900	0.6862	-0.2097	-2.3411	0.7073	2.1315	-49.4707	-37.4653	-2.2482	-2.4234
0.0572	2.7507	1920	0.6807	-0.0767	-2.3365	0.7256	2.2598	-49.4668	-37.3545	-2.2578	-2.4338
0.1091	2.7794	1940	0.6759	0.0803	-2.1670	0.7622	2.2473	-49.3255	-37.2236	-2.2632	-2.4376
0.1232	2.8080	1960	0.6741	0.0332	-2.2451	0.7378	2.2783	-49.3906	-37.2629	-2.2666	-2.4412
0.0923	2.8367	1980	0.6796	-0.0260	-2.2976	0.75	2.2716	-49.4344	-37.3122	-2.2677	-2.4426
0.0896	2.8653	2000	0.6839	-0.1227	-2.3817	0.7317	2.2590	-49.5045	-37.3928	-2.2640	-2.4394
0.2387	2.8940	2020	0.6769	-0.0674	-2.3951	0.7439	2.3277	-49.5157	-37.3468	-2.2608	-2.4367
0.3573	2.9226	2040	0.6774	-0.1044	-2.3942	0.7622	2.2899	-49.5149	-37.3776	-2.2517	-2.4269
0.3164	2.9513	2060	0.6741	-0.1523	-2.5126	0.7378	2.3604	-49.6136	-37.4174	-2.2540	-2.4306
0.3174	2.9799	2080	0.6767	-0.1774	-2.5571	0.75	2.3798	-49.6506	-37.4384	-2.2527	-2.4293
0.0088	3.0086	2100	0.6752	-0.1327	-2.4041	0.7439	2.2714	-49.5231	-37.4012	-2.2539	-2.4300
0.0236	3.0372	2120	0.6855	-0.2512	-2.5781	0.7317	2.3269	-49.6681	-37.4999	-2.2529	-2.4309
0.0103	3.0659	2140	0.7067	0.1106	-2.2651	0.7439	2.3757	-49.4073	-37.1984	-2.2918	-2.4702
0.3669	3.0946	2160	0.7251	0.0983	-2.4624	0.7622	2.5607	-49.5717	-37.2086	-2.2949	-2.4748
0.0092	3.1232	2180	0.7444	0.1080	-2.5510	0.7317	2.6590	-49.6455	-37.2006	-2.3042	-2.4842
0.018	3.1519	2200	0.7438	0.0810	-2.6245	0.7317	2.7055	-49.7068	-37.2231	-2.3008	-2.4814
0.0108	3.1805	2220	0.7442	0.2002	-2.5832	0.7439	2.7834	-49.6724	-37.1237	-2.3121	-2.4934
0.0936	3.2092	2240	0.7552	0.0517	-2.7181	0.75	2.7697	-49.7848	-37.2475	-2.3113	-2.4914
0.017	3.2378	2260	0.7712	0.1502	-2.7387	0.7561	2.8888	-49.8019	-37.1654	-2.3257	-2.5076
0.0192	3.2665	2280	0.7523	0.1559	-2.7291	0.7439	2.8850	-49.7940	-37.1606	-2.3249	-2.5065
0.0039	3.2951	2300	0.7512	-0.1228	-3.0815	0.7622	2.9587	-50.0876	-37.3929	-2.3222	-2.5044
0.0072	3.3238	2320	0.7550	-0.1381	-2.9920	0.7683	2.8540	-50.0131	-37.4056	-2.3160	-2.4983
0.0093	3.3524	2340	0.7604	-0.2021	-3.1653	0.7744	2.9632	-50.1575	-37.4590	-2.3162	-2.4983
0.0083	3.3811	2360	0.7673	-0.1799	-3.1950	0.7561	3.0151	-50.1822	-37.4405	-2.3246	-2.5065
0.0935	3.4097	2380	0.7630	0.0108	-3.0077	0.7805	3.0184	-50.0261	-37.2816	-2.3351	-2.5182
0.0166	3.4384	2400	0.7579	-0.0949	-3.1146	0.7744	3.0197	-50.1152	-37.3697	-2.3268	-2.5100
0.0298	3.4670	2420	0.7513	-0.3619	-3.4170	0.7683	3.0550	-50.3672	-37.5922	-2.3079	-2.4910
0.0352	3.4957	2440	0.7575	-0.4634	-3.5102	0.7683	3.0468	-50.4449	-37.6768	-2.3024	-2.4868
0.0309	3.5244	2460	0.7682	-0.3554	-3.4486	0.7439	3.0931	-50.3935	-37.5868	-2.3085	-2.4932
0.0196	3.5530	2480	0.7856	-0.3255	-3.4376	0.7378	3.1121	-50.3844	-37.5618	-2.3231	-2.5089
0.0227	3.5817	2500	0.7959	-0.3269	-3.3548	0.7317	3.0279	-50.3154	-37.5630	-2.3282	-2.5152
0.0195	3.6103	2520	0.7943	-0.4678	-3.5794	0.75	3.1116	-50.5026	-37.6804	-2.3099	-2.4960
0.0351	3.6390	2540	0.7861	-0.5632	-3.7540	0.7378	3.1908	-50.6480	-37.7599	-2.3047	-2.4900
0.1676	3.6676	2560	0.7739	-0.2552	-3.4376	0.7378	3.1824	-50.3844	-37.5033	-2.3324	-2.5184
0.003	3.6963	2580	0.7878	-0.1548	-3.2436	0.7378	3.0888	-50.2227	-37.4195	-2.3376	-2.5225
0.064	3.7249	2600	0.7744	-0.1345	-3.2564	0.7256	3.1219	-50.2334	-37.4027	-2.3374	-2.5238
0.0316	3.7536	2620	0.7811	-0.3624	-3.5206	0.7622	3.1582	-50.4535	-37.5925	-2.3162	-2.5026
0.1376	3.7822	2640	0.7919	-0.3476	-3.5977	0.7561	3.2501	-50.5178	-37.5802	-2.3183	-2.5040
0.0037	3.8109	2660	0.7925	-0.1196	-3.2857	0.7866	3.1662	-50.2578	-37.3902	-2.3428	-2.5286
0.0371	3.8395	2680	0.7962	-0.0187	-3.2665	0.7561	3.2478	-50.2418	-37.3062	-2.3491	-2.5358
0.0109	3.8682	2700	0.7986	-0.2099	-3.3888	0.7683	3.1789	-50.3437	-37.4655	-2.3390	-2.5267
0.074	3.8968	2720	0.7970	-0.0908	-3.2281	0.7256	3.1373	-50.2098	-37.3662	-2.3424	-2.5295
0.019	3.9255	2740	0.7908	-0.2582	-3.4584	0.7439	3.2002	-50.4017	-37.5058	-2.3421	-2.5290
0.1003	3.9542	2760	0.7889	-0.3895	-3.5896	0.7561	3.2001	-50.5110	-37.6151	-2.3338	-2.5209
0.0136	3.9828	2780	0.7855	-0.3688	-3.6660	0.7439	3.2972	-50.5747	-37.5979	-2.3322	-2.5192
0.0024	4.0115	2800	0.8003	-0.3754	-3.7187	0.7683	3.3433	-50.6186	-37.6034	-2.3462	-2.5336
0.0101	4.0401	2820	0.8140	-0.2934	-3.6804	0.7317	3.3870	-50.5867	-37.5351	-2.3586	-2.5461
0.0044	4.0688	2840	0.8276	-0.3098	-3.8388	0.7317	3.5290	-50.7187	-37.5488	-2.3619	-2.5503
0.0026	4.0974	2860	0.8373	-0.3348	-3.8660	0.7622	3.5311	-50.7413	-37.5696	-2.3634	-2.5516
0.003	4.1261	2880	0.8305	-0.5823	-4.1652	0.7439	3.5829	-50.9907	-37.7758	-2.3566	-2.5453
0.0026	4.1547	2900	0.8375	-0.4957	-4.1598	0.7622	3.6641	-50.9862	-37.7036	-2.3654	-2.5553
0.0079	4.1834	2920	0.8436	-0.6782	-4.3336	0.7378	3.6554	-51.1310	-37.8557	-2.3636	-2.5548
0.0027	4.2120	2940	0.8560	-0.4767	-4.1517	0.7378	3.6750	-50.9794	-37.6878	-2.3784	-2.5694
0.0014	4.2407	2960	0.8595	-0.4925	-4.2464	0.7561	3.7539	-51.0584	-37.7010	-2.3825	-2.5744
0.0013	4.2693	2980	0.8613	-0.4685	-4.2737	0.7561	3.8052	-51.0811	-37.6810	-2.3933	-2.5848
0.0142	4.2980	3000	0.8828	-0.3902	-4.2718	0.75	3.8817	-51.0796	-37.6157	-2.4055	-2.5962
0.0055	4.3266	3020	0.8840	-0.4200	-4.3977	0.7439	3.9778	-51.1845	-37.6405	-2.4073	-2.5978
0.1919	4.3553	3040	0.8941	-0.5490	-4.5609	0.7744	4.0119	-51.3204	-37.7481	-2.4005	-2.5923
0.0026	4.3840	3060	0.8871	-0.6962	-4.7043	0.7622	4.0081	-51.4399	-37.8708	-2.3919	-2.5833
0.0023	4.4126	3080	0.8967	-0.8819	-4.8442	0.7622	3.9623	-51.5565	-38.0255	-2.3890	-2.5808
0.0026	4.4413	3100	0.9085	-0.6495	-4.7969	0.7622	4.1474	-51.5171	-37.8318	-2.4106	-2.6033
0.0033	4.4699	3120	0.9194	-0.4728	-4.6496	0.7561	4.1768	-51.3943	-37.6846	-2.4287	-2.6203
0.0027	4.4986	3140	0.9166	-0.5691	-4.7933	0.7622	4.2242	-51.5141	-37.7648	-2.4242	-2.6161
0.0018	4.5272	3160	0.9134	-0.5995	-4.7722	0.7561	4.1728	-51.4966	-37.7901	-2.4213	-2.6125
0.0121	4.5559	3180	0.9177	-0.5039	-4.6475	0.7622	4.1436	-51.3926	-37.7105	-2.4304	-2.6226
0.0012	4.5845	3200	0.9261	-0.5518	-4.7423	0.7683	4.1905	-51.4716	-37.7504	-2.4260	-2.6186
0.0017	4.6132	3220	0.9241	-0.6581	-4.8877	0.7744	4.2297	-51.5928	-37.8390	-2.4256	-2.6191
0.0024	4.6418	3240	0.9358	-0.6434	-4.8042	0.7561	4.1608	-51.5232	-37.8268	-2.4263	-2.6198
0.0011	4.6705	3260	0.9427	-0.6950	-4.7806	0.7561	4.0856	-51.5035	-37.8697	-2.4222	-2.6148
0.2681	4.6991	3280	0.9378	-0.5942	-4.8087	0.7683	4.2145	-51.5270	-37.7858	-2.4248	-2.6185
0.0232	4.7278	3300	0.9449	-0.4714	-4.7353	0.7561	4.2640	-51.4658	-37.6834	-2.4304	-2.6240
0.0014	4.7564	3320	0.9316	-0.7614	-4.9840	0.7622	4.2226	-51.6731	-37.9251	-2.4226	-2.6165
0.0038	4.7851	3340	0.9506	-0.8514	-5.0513	0.7561	4.1998	-51.7291	-38.0001	-2.4223	-2.6166
0.0019	4.8138	3360	0.9394	-0.7083	-4.9715	0.7561	4.2632	-51.6626	-37.8808	-2.4256	-2.6194
0.0072	4.8424	3380	0.9570	-0.7356	-4.9754	0.7439	4.2398	-51.6658	-37.9035	-2.4270	-2.6209
0.0042	4.8711	3400	0.9600	-0.7845	-5.0574	0.7683	4.2729	-51.7342	-37.9443	-2.4284	-2.6220
0.0083	4.8997	3420	0.9460	-0.7969	-5.0709	0.7622	4.2740	-51.7454	-37.9546	-2.4237	-2.6173
0.0017	4.9284	3440	0.9523	-0.7037	-4.9795	0.7561	4.2758	-51.6693	-37.8770	-2.4335	-2.6282
0.0007	4.9570	3460	0.9418	-0.7546	-5.0488	0.7561	4.2942	-51.7270	-37.9194	-2.4297	-2.6232
0.0051	4.9857	3480	0.9424	-0.8220	-5.0635	0.7683	4.2416	-51.7393	-37.9755	-2.4272	-2.6208
0.0009	5.0143	3500	0.9510	-0.8179	-5.0827	0.7744	4.2648	-51.7553	-37.9722	-2.4302	-2.6247
0.0003	5.0430	3520	0.9454	-0.8076	-5.2205	0.7866	4.4129	-51.8701	-37.9636	-2.4330	-2.6273
0.001	5.0716	3540	0.9462	-0.8147	-5.1976	0.7622	4.3828	-51.8510	-37.9695	-2.4411	-2.6357
0.0079	5.1003	3560	0.9502	-0.8524	-5.2850	0.7744	4.4326	-51.9239	-38.0009	-2.4462	-2.6400
0.0005	5.1289	3580	0.9660	-0.9093	-5.3630	0.7866	4.4538	-51.9889	-38.0483	-2.4495	-2.6439
0.0003	5.1576	3600	0.9780	-0.8057	-5.4083	0.7744	4.6026	-52.0266	-37.9620	-2.4601	-2.6544
0.0006	5.1862	3620	0.9789	-0.8859	-5.4701	0.7805	4.5841	-52.0781	-38.0289	-2.4599	-2.6539
0.0006	5.2149	3640	0.9848	-0.8710	-5.4639	0.7683	4.5930	-52.0730	-38.0164	-2.4638	-2.6583
0.0004	5.2436	3660	0.9941	-1.0250	-5.6095	0.7683	4.5844	-52.1943	-38.1448	-2.4636	-2.6588
0.0006	5.2722	3680	1.0063	-1.0677	-5.8045	0.7805	4.7368	-52.3568	-38.1803	-2.4596	-2.6542
0.0008	5.3009	3700	1.0097	-1.1606	-5.8185	0.7439	4.6579	-52.3685	-38.2578	-2.4610	-2.6560
0.0022	5.3295	3720	0.9950	-1.1802	-5.8888	0.7805	4.7086	-52.4270	-38.2740	-2.4587	-2.6544
0.0005	5.3582	3740	1.0147	-1.1293	-5.8909	0.7744	4.7616	-52.4288	-38.2317	-2.4649	-2.6595
0.0023	5.3868	3760	1.0118	-1.0642	-5.7499	0.7622	4.6857	-52.3113	-38.1774	-2.4737	-2.6677
0.0088	5.4155	3780	1.0252	-0.9390	-5.7444	0.7561	4.8054	-52.3067	-38.0731	-2.4777	-2.6726
0.0001	5.4441	3800	1.0191	-0.9007	-5.7038	0.7561	4.8031	-52.2729	-38.0412	-2.4823	-2.6773
0.0003	5.4728	3820	1.0239	-0.9196	-5.7256	0.7561	4.8060	-52.2911	-38.0569	-2.4836	-2.6781
0.0016	5.5014	3840	1.0421	-0.9575	-5.7242	0.7622	4.7667	-52.2898	-38.0885	-2.4864	-2.6819
0.0002	5.5301	3860	1.0487	-1.0003	-5.8026	0.7622	4.8023	-52.3552	-38.1242	-2.4898	-2.6855
0.0004	5.5587	3880	1.0393	-1.0408	-5.8395	0.7622	4.7987	-52.3859	-38.1579	-2.4752	-2.6700
0.0005	5.5874	3900	1.0304	-1.0372	-5.9509	0.7561	4.9137	-52.4788	-38.1549	-2.4765	-2.6715
0.0009	5.6160	3920	1.0386	-1.1627	-5.9480	0.7439	4.7852	-52.4764	-38.2595	-2.4782	-2.6741
0.0148	5.6447	3940	1.0307	-1.0947	-6.0190	0.7439	4.9244	-52.5356	-38.2028	-2.4858	-2.6811
0.001	5.6734	3960	1.0436	-1.1245	-6.0623	0.7622	4.9377	-52.5716	-38.2277	-2.4882	-2.6832
0.0027	5.7020	3980	1.0612	-1.0429	-5.9853	0.7439	4.9423	-52.5074	-38.1597	-2.4948	-2.6906
0.0012	5.7307	4000	1.0467	-1.1036	-6.0613	0.7805	4.9577	-52.5708	-38.2103	-2.4925	-2.6887
0.001	5.7593	4020	1.0433	-1.0793	-6.1130	0.7622	5.0337	-52.6139	-38.1900	-2.4900	-2.6860
0.002	5.7880	4040	1.0448	-1.0374	-6.0253	0.7683	4.9879	-52.5408	-38.1551	-2.4916	-2.6884
0.0005	5.8166	4060	1.0504	-0.9918	-6.0202	0.75	5.0284	-52.5365	-38.1170	-2.4938	-2.6901
0.001	5.8453	4080	1.0608	-0.9703	-6.0440	0.7683	5.0737	-52.5564	-38.0992	-2.4966	-2.6920
0.0005	5.8739	4100	1.0614	-1.0198	-6.0760	0.75	5.0562	-52.5831	-38.1404	-2.5001	-2.6965
0.0004	5.9026	4120	1.0605	-1.0490	-6.0863	0.7439	5.0373	-52.5916	-38.1648	-2.5006	-2.6968
0.0003	5.9312	4140	1.0487	-1.0157	-6.1166	0.75	5.1009	-52.6169	-38.1370	-2.4927	-2.6891
0.0012	5.9599	4160	1.0663	-1.1262	-6.2213	0.7439	5.0951	-52.7041	-38.2291	-2.4933	-2.6902
0.0005	5.9885	4180	1.0671	-1.1373	-6.2442	0.7561	5.1069	-52.7232	-38.2384	-2.5015	-2.7005
0.0004	6.0172	4200	1.0697	-1.1305	-6.2184	0.75	5.0879	-52.7017	-38.2327	-2.4971	-2.6941
0.0001	6.0458	4220	1.0760	-1.0741	-6.1588	0.75	5.0847	-52.6521	-38.1857	-2.4996	-2.6965
0.0006	6.0745	4240	1.0792	-1.0182	-6.2045	0.7683	5.1863	-52.6901	-38.1390	-2.5082	-2.7050
0.0004	6.1032	4260	1.0726	-1.0702	-6.2083	0.7561	5.1381	-52.6933	-38.1824	-2.5066	-2.7029
0.0002	6.1318	4280	1.0755	-1.0338	-6.1528	0.7744	5.1190	-52.6470	-38.1521	-2.5107	-2.7069
0.0002	6.1605	4300	1.0740	-0.9697	-6.1554	0.7683	5.1857	-52.6492	-38.0987	-2.5147	-2.7110
0.0002	6.1891	4320	1.0964	-1.0407	-6.2262	0.75	5.1854	-52.7082	-38.1579	-2.5200	-2.7169
0.0004	6.2178	4340	1.0731	-0.9927	-6.2377	0.7622	5.2449	-52.7178	-38.1179	-2.5123	-2.7077
0.0003	6.2464	4360	1.0885	-1.0033	-6.2182	0.7683	5.2149	-52.7016	-38.1267	-2.5165	-2.7132
0.0001	6.2751	4380	1.0898	-1.1023	-6.2981	0.7805	5.1958	-52.7681	-38.2091	-2.5177	-2.7148
0.0002	6.3037	4400	1.0767	-1.1102	-6.3390	0.7683	5.2288	-52.8022	-38.2158	-2.5222	-2.7201
0.0003	6.3324	4420	1.0850	-1.0746	-6.3758	0.7805	5.3012	-52.8329	-38.1861	-2.5179	-2.7150
0.0001	6.3610	4440	1.0943	-1.0926	-6.3811	0.7866	5.2885	-52.8373	-38.2011	-2.5173	-2.7137
0.0002	6.3897	4460	1.0893	-1.1349	-6.4395	0.7683	5.3045	-52.8859	-38.2363	-2.5214	-2.7188
0.0004	6.4183	4480	1.0977	-1.0691	-6.4414	0.7744	5.3723	-52.8875	-38.1815	-2.5215	-2.7180
0.0001	6.4470	4500	1.0958	-1.1687	-6.4625	0.7561	5.2938	-52.9051	-38.2645	-2.5159	-2.7129
0.0002	6.4756	4520	1.1056	-1.1135	-6.4737	0.7561	5.3602	-52.9145	-38.2185	-2.5169	-2.7135
0.0003	6.5043	4540	1.1006	-1.1675	-6.4397	0.7744	5.2722	-52.8861	-38.2635	-2.5174	-2.7137
0.0004	6.5330	4560	1.0982	-1.0982	-6.4121	0.75	5.3139	-52.8632	-38.2058	-2.5245	-2.7222
0.0015	6.5616	4580	1.0962	-1.1336	-6.4864	0.7683	5.3528	-52.9251	-38.2352	-2.5221	-2.7190
0.0001	6.5903	4600	1.1073	-1.1322	-6.4663	0.7561	5.3341	-52.9083	-38.2341	-2.5228	-2.7192
0.0002	6.6189	4620	1.1041	-1.1144	-6.4878	0.7622	5.3734	-52.9262	-38.2192	-2.5317	-2.7298
0.0001	6.6476	4640	1.1062	-1.0988	-6.4521	0.7683	5.3533	-52.8964	-38.2062	-2.5288	-2.7263
0.0002	6.6762	4660	1.1063	-1.1378	-6.4561	0.7683	5.3183	-52.8998	-38.2387	-2.5252	-2.7217
0.0001	6.7049	4680	1.0989	-1.1028	-6.5373	0.7866	5.4345	-52.9674	-38.2096	-2.5254	-2.7226
0.0003	6.7335	4700	1.1133	-1.1656	-6.4323	0.75	5.2667	-52.8800	-38.2619	-2.5255	-2.7225
0.0001	6.7622	4720	1.1066	-1.1477	-6.4661	0.7805	5.3184	-52.9081	-38.2470	-2.5238	-2.7207
0.0002	6.7908	4740	1.0956	-1.1597	-6.5258	0.7683	5.3662	-52.9579	-38.2570	-2.5226	-2.7191
0.0004	6.8195	4760	1.1079	-1.1313	-6.4817	0.7622	5.3503	-52.9211	-38.2334	-2.5251	-2.7217
0.0017	6.8481	4780	1.1020	-1.1242	-6.5045	0.7744	5.3803	-52.9402	-38.2274	-2.5301	-2.7278
0.0004	6.8768	4800	1.1095	-1.1847	-6.5202	0.7439	5.3355	-52.9532	-38.2778	-2.5219	-2.7185
0.0002	6.9054	4820	1.0962	-1.1130	-6.5307	0.7866	5.4177	-52.9620	-38.2181	-2.5261	-2.7238
0.0001	6.9341	4840	1.1102	-1.1634	-6.5876	0.7561	5.4241	-53.0094	-38.2601	-2.5275	-2.7252
0.0004	6.9628	4860	1.1080	-1.1609	-6.4743	0.7744	5.3134	-52.9150	-38.2580	-2.5233	-2.7201
0.0006	6.9914	4880	1.0967	-1.1542	-6.5908	0.7683	5.4366	-53.0121	-38.2524	-2.5262	-2.7232
0.0001	7.0201	4900	1.1093	-1.2228	-6.5951	0.7744	5.3723	-53.0156	-38.3096	-2.5253	-2.7228
0.0003	7.0487	4920	1.1029	-1.1754	-6.5485	0.7744	5.3732	-52.9768	-38.2701	-2.5293	-2.7269
0.0001	7.0774	4940	1.1111	-1.1697	-6.5618	0.7683	5.3921	-52.9879	-38.2653	-2.5237	-2.7206
0.0002	7.1060	4960	1.1108	-1.2530	-6.5827	0.7622	5.3297	-53.0053	-38.3347	-2.5302	-2.7288
0.0001	7.1347	4980	1.1181	-1.2157	-6.5811	0.7561	5.3655	-53.0040	-38.3036	-2.5312	-2.7292
0.0002	7.1633	5000	1.1051	-1.1687	-6.6222	0.7622	5.4534	-53.0382	-38.2645	-2.5315	-2.7291
0.0001	7.1920	5020	1.1066	-1.2478	-6.6124	0.7683	5.3646	-53.0300	-38.3304	-2.5276	-2.7243
0.0002	7.2206	5040	1.1089	-1.1568	-6.5504	0.7561	5.3936	-52.9784	-38.2545	-2.5282	-2.7252
0.0001	7.2493	5060	1.1123	-1.2366	-6.5455	0.7622	5.3089	-52.9743	-38.3210	-2.5296	-2.7265
0.0002	7.2779	5080	1.1058	-1.1717	-6.5729	0.7683	5.4012	-52.9971	-38.2670	-2.5259	-2.7227
0.0	7.3066	5100	1.1127	-1.1795	-6.6398	0.7866	5.4602	-53.0528	-38.2735	-2.5298	-2.7268
0.0002	7.3352	5120	1.1148	-1.2084	-6.5895	0.75	5.3811	-53.0109	-38.2976	-2.5264	-2.7235
0.0002	7.3639	5140	1.1143	-1.1701	-6.6157	0.7622	5.4456	-53.0328	-38.2657	-2.5325	-2.7303
0.0002	7.3926	5160	1.1073	-1.1712	-6.5088	0.75	5.3376	-52.9437	-38.2666	-2.5319	-2.7298
0.0001	7.4212	5180	1.1226	-1.2326	-6.6428	0.7561	5.4102	-53.0553	-38.3177	-2.5294	-2.7265
0.0002	7.4499	5200	1.1198	-1.1555	-6.5387	0.7744	5.3832	-52.9686	-38.2535	-2.5296	-2.7266
0.0002	7.4785	5220	1.1119	-1.2103	-6.6066	0.75	5.3963	-53.0252	-38.2992	-2.5324	-2.7307
0.0002	7.5072	5240	1.1072	-1.1657	-6.6662	0.7927	5.5005	-53.0749	-38.2620	-2.5298	-2.7271
0.0002	7.5358	5260	1.1150	-1.1280	-6.5967	0.7683	5.4687	-53.0170	-38.2306	-2.5319	-2.7296
0.0005	7.5645	5280	1.0950	-1.2590	-6.6428	0.7744	5.3837	-53.0553	-38.3398	-2.5297	-2.7270
0.0001	7.5931	5300	1.1019	-1.1696	-6.5989	0.7622	5.4293	-53.0188	-38.2652	-2.5339	-2.7316
0.0002	7.6218	5320	1.0991	-1.2744	-6.6140	0.75	5.3396	-53.0314	-38.3526	-2.5305	-2.7281
0.0002	7.6504	5340	1.1187	-1.2180	-6.5979	0.7805	5.3799	-53.0180	-38.3056	-2.5328	-2.7313
0.0001	7.6791	5360	1.1139	-1.2194	-6.6002	0.7622	5.3808	-53.0198	-38.3067	-2.5264	-2.7234
0.0002	7.7077	5380	1.1102	-1.2205	-6.5572	0.75	5.3367	-52.9840	-38.3076	-2.5303	-2.7279
0.0003	7.7364	5400	1.1330	-1.2218	-6.5822	0.7561	5.3604	-53.0049	-38.3088	-2.5299	-2.7272
0.0001	7.7650	5420	1.1238	-1.1740	-6.6320	0.7683	5.4580	-53.0464	-38.2689	-2.5340	-2.7318
0.0001	7.7937	5440	1.1161	-1.2118	-6.6073	0.7805	5.3955	-53.0258	-38.3004	-2.5330	-2.7313
0.0002	7.8223	5460	1.1162	-1.1727	-6.5552	0.7866	5.3824	-52.9823	-38.2678	-2.5302	-2.7278
0.0003	7.8510	5480	1.1124	-1.1982	-6.5749	0.7683	5.3766	-52.9988	-38.2891	-2.5333	-2.7308
0.0001	7.8797	5500	1.1055	-1.2175	-6.6250	0.7683	5.4075	-53.0405	-38.3051	-2.5337	-2.7316
0.0001	7.9083	5520	1.1062	-1.2122	-6.6535	0.75	5.4413	-53.0643	-38.3007	-2.5328	-2.7304
0.0001	7.9370	5540	1.1325	-1.2330	-6.5237	0.75	5.2907	-52.9562	-38.3181	-2.5334	-2.7309
0.0001	7.9656	5560	1.1335	-1.1828	-6.6097	0.7683	5.4269	-53.0278	-38.2762	-2.5330	-2.7307
0.0014	7.9943	5580	1.1112	-1.1799	-6.6516	0.7622	5.4716	-53.0627	-38.2739	-2.5328	-2.7308

Framework versions

Transformers 4.45.2
Pytorch 2.5.1+cu121
Datasets 3.5.0
Tokenizers 0.20.3

YuchenLi01/genSoftQwen2.5MathRM72Bth0.5pair4NoGT_1.5B_dpo_ebs32_lr5e-07_beta1.2_epoch8.0_42

genSoftQwen2.5MathRM72Bth0.5pair4NoGT_1.5B_dpo_ebs32_lr5e-07_beta1.2_epoch8.0_42

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

YuchenLi01/
genSoftQwen2.5MathRM72Bth0.5pair4NoGT_1.5B_dpo_ebs32_lr5e-07_beta1.2_epoch8.0_42