This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_Soft_DPO_Qwen2.5MathRM72B_th0.5_MoreNoGT dataset. It achieves the following results on the evaluation set:

Loss: 0.5083
Rewards/chosen: -0.8960
Rewards/rejected: -2.1022
Rewards/accuracies: 0.7215
Rewards/margins: 1.2061
Logps/rejected: -65.4200
Logps/chosen: -51.0221
Logits/rejected: -2.1804
Logits/chosen: -2.3166

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6567	0.0057	20	0.6975	-0.0105	-0.0124	0.5198	0.0018	-60.1954	-48.8083	-2.1735	-2.2745
0.7086	0.0113	40	0.6969	-0.0049	-0.0080	0.4938	0.0031	-60.1844	-48.7942	-2.1718	-2.2728
0.6846	0.0170	60	0.6973	-0.0004	-0.0180	0.5322	0.0176	-60.2095	-48.7830	-2.1696	-2.2706
0.6891	0.0226	80	0.6961	-0.0135	-0.0205	0.5260	0.0070	-60.2159	-48.8158	-2.1731	-2.2744
0.7258	0.0283	100	0.6983	-0.0110	-0.0098	0.4926	-0.0012	-60.1891	-48.8094	-2.1730	-2.2742
0.6738	0.0339	120	0.6957	-0.0140	-0.0187	0.5074	0.0047	-60.2114	-48.8171	-2.1736	-2.2749
0.7126	0.0396	140	0.6928	-0.0143	-0.0285	0.5309	0.0142	-60.2358	-48.8177	-2.1693	-2.2706
0.6797	0.0452	160	0.6911	-0.0239	-0.0418	0.5111	0.0179	-60.2690	-48.8418	-2.1692	-2.2709
0.6984	0.0509	180	0.6875	-0.0537	-0.0633	0.5297	0.0097	-60.3229	-48.9162	-2.1668	-2.2693
0.6882	0.0565	200	0.6824	-0.0879	-0.1298	0.5743	0.0419	-60.4891	-49.0018	-2.1520	-2.2543
0.6778	0.0622	220	0.6793	-0.0952	-0.1267	0.5495	0.0315	-60.4813	-49.0201	-2.1585	-2.2620
0.6677	0.0678	240	0.6733	-0.1074	-0.1566	0.5631	0.0492	-60.5559	-49.0504	-2.1528	-2.2567
0.7111	0.0735	260	0.6696	-0.1170	-0.1843	0.5755	0.0673	-60.6252	-49.0745	-2.1584	-2.2638
0.6185	0.0791	280	0.6646	-0.1633	-0.2485	0.5990	0.0852	-60.7858	-49.1903	-2.1433	-2.2490
0.7275	0.0848	300	0.6589	-0.1857	-0.2866	0.6139	0.1009	-60.8810	-49.2461	-2.1427	-2.2500
0.6771	0.0904	320	0.6510	-0.1681	-0.2873	0.6460	0.1192	-60.8828	-49.2023	-2.1503	-2.2590
0.6266	0.0961	340	0.6448	-0.1307	-0.2819	0.6262	0.1512	-60.8692	-49.1087	-2.1662	-2.2761
0.6214	0.1017	360	0.6383	-0.1833	-0.3639	0.6411	0.1806	-61.0742	-49.2402	-2.1610	-2.2722
0.6333	0.1074	380	0.6321	-0.1975	-0.4172	0.6312	0.2197	-61.2076	-49.2758	-2.1624	-2.2748
0.6711	0.1130	400	0.6285	-0.2354	-0.4749	0.6535	0.2394	-61.3517	-49.3706	-2.1569	-2.2705
0.5723	0.1187	420	0.6208	-0.1664	-0.4308	0.6584	0.2644	-61.2415	-49.1980	-2.1769	-2.2917
0.6851	0.1243	440	0.6151	-0.1805	-0.4762	0.6597	0.2957	-61.3551	-49.2332	-2.1800	-2.2964
0.7111	0.1300	460	0.6102	-0.1696	-0.4677	0.6547	0.2981	-61.3338	-49.2059	-2.1829	-2.2996
0.5466	0.1356	480	0.6080	-0.2522	-0.5826	0.6832	0.3303	-61.6209	-49.4125	-2.1683	-2.2860
0.6134	0.1413	500	0.6022	-0.2888	-0.6432	0.6832	0.3544	-61.7724	-49.5040	-2.1664	-2.2851
0.6037	0.1469	520	0.6019	-0.3636	-0.7369	0.6658	0.3733	-62.0068	-49.6911	-2.1484	-2.2671
0.6135	0.1526	540	0.5975	-0.3885	-0.7881	0.6832	0.3996	-62.1347	-49.7532	-2.1475	-2.2665
0.5761	0.1582	560	0.5963	-0.3804	-0.7866	0.6819	0.4061	-62.1309	-49.7331	-2.1497	-2.2692
0.5531	0.1639	580	0.5937	-0.4190	-0.8464	0.6881	0.4274	-62.2804	-49.8294	-2.1443	-2.2648
0.4641	0.1695	600	0.5895	-0.4999	-0.9558	0.6918	0.4559	-62.5540	-50.0318	-2.1268	-2.2478
0.7448	0.1752	620	0.5847	-0.4903	-1.0024	0.7017	0.5122	-62.6706	-50.0077	-2.1387	-2.2610
0.6814	0.1808	640	0.5826	-0.4193	-0.9590	0.7005	0.5397	-62.5620	-49.8302	-2.1602	-2.2831
0.8746	0.1865	660	0.5791	-0.4203	-0.9563	0.6931	0.5361	-62.5554	-49.8327	-2.1645	-2.2874
0.5546	0.1921	680	0.5741	-0.3074	-0.8659	0.6844	0.5585	-62.3293	-49.5505	-2.1817	-2.3048
0.6043	0.1978	700	0.5711	-0.3879	-0.9738	0.6968	0.5859	-62.5990	-49.7518	-2.1687	-2.2925
0.4609	0.2034	720	0.5684	-0.3508	-0.9525	0.6918	0.6017	-62.5457	-49.6589	-2.1799	-2.3043
0.4435	0.2091	740	0.5666	-0.3143	-0.9050	0.6968	0.5906	-62.4269	-49.5678	-2.1881	-2.3127
0.5202	0.2147	760	0.5624	-0.3238	-0.9268	0.6906	0.6030	-62.4815	-49.5915	-2.1896	-2.3143
0.5555	0.2204	780	0.5640	-0.3151	-0.9340	0.6856	0.6188	-62.4995	-49.5698	-2.1951	-2.3200
0.5387	0.2261	800	0.5591	-0.3894	-1.0351	0.6906	0.6457	-62.7523	-49.7555	-2.1850	-2.3106
0.5375	0.2317	820	0.5571	-0.4007	-1.0659	0.6931	0.6652	-62.8292	-49.7836	-2.1856	-2.3111
0.6226	0.2374	840	0.5544	-0.3949	-1.0785	0.7104	0.6836	-62.8608	-49.7692	-2.1888	-2.3141
0.5045	0.2430	860	0.5536	-0.4071	-1.1019	0.7067	0.6948	-62.9192	-49.7998	-2.1977	-2.3240
0.6512	0.2487	880	0.5508	-0.4764	-1.2199	0.7042	0.7435	-63.2144	-49.9731	-2.1823	-2.3088
0.6313	0.2543	900	0.5504	-0.4684	-1.2120	0.7252	0.7436	-63.1945	-49.9530	-2.1735	-2.3004
0.5781	0.2600	920	0.5474	-0.4549	-1.1908	0.7079	0.7359	-63.1414	-49.9192	-2.1693	-2.2956
0.5172	0.2656	940	0.5455	-0.3703	-1.1158	0.7017	0.7455	-62.9540	-49.7077	-2.1961	-2.3226
0.5788	0.2713	960	0.5432	-0.4939	-1.2831	0.7191	0.7892	-63.3722	-50.0167	-2.1826	-2.3108
0.6007	0.2769	980	0.5400	-0.5622	-1.3642	0.7116	0.8020	-63.5751	-50.1876	-2.1649	-2.2928
0.5809	0.2826	1000	0.5381	-0.5176	-1.3146	0.6980	0.7970	-63.4510	-50.0761	-2.1724	-2.3011
0.5882	0.2882	1020	0.5364	-0.6069	-1.4314	0.7166	0.8245	-63.7430	-50.2992	-2.1517	-2.2805
0.5801	0.2939	1040	0.5349	-0.6174	-1.4353	0.7129	0.8179	-63.7529	-50.3255	-2.1504	-2.2783
0.4942	0.2995	1060	0.5326	-0.6298	-1.4797	0.7104	0.8499	-63.8637	-50.3565	-2.1475	-2.2754
0.4813	0.3052	1080	0.5324	-0.6550	-1.4932	0.7104	0.8382	-63.8975	-50.4194	-2.1353	-2.2633
0.5376	0.3108	1100	0.5330	-0.7038	-1.5645	0.7092	0.8608	-64.0759	-50.5414	-2.1338	-2.2624
0.5811	0.3165	1120	0.5311	-0.7746	-1.6597	0.7042	0.8851	-64.3137	-50.7185	-2.1235	-2.2527
0.557	0.3221	1140	0.5330	-0.7656	-1.6716	0.7178	0.9060	-64.3435	-50.6959	-2.1303	-2.2594
0.5329	0.3278	1160	0.5282	-0.7576	-1.6905	0.7228	0.9329	-64.3908	-50.6759	-2.1345	-2.2632
0.4985	0.3334	1180	0.5273	-0.8069	-1.7551	0.7228	0.9482	-64.5522	-50.7993	-2.1319	-2.2609
0.4422	0.3391	1200	0.5280	-0.8153	-1.7598	0.7178	0.9445	-64.5639	-50.8201	-2.1328	-2.2619
0.5554	0.3447	1220	0.5271	-0.6902	-1.6048	0.7116	0.9146	-64.1765	-50.5075	-2.1514	-2.2802
0.5047	0.3504	1240	0.5262	-0.6535	-1.5709	0.7228	0.9174	-64.0917	-50.4157	-2.1529	-2.2821
0.5021	0.3560	1260	0.5276	-0.6305	-1.5530	0.7104	0.9225	-64.0471	-50.3583	-2.1610	-2.2908
0.424	0.3617	1280	0.5252	-0.6985	-1.6380	0.7141	0.9395	-64.2596	-50.5283	-2.1476	-2.2771
0.4083	0.3673	1300	0.5252	-0.7039	-1.6749	0.7054	0.9711	-64.3519	-50.5416	-2.1623	-2.2921
0.5785	0.3730	1320	0.5234	-0.7097	-1.7139	0.7005	1.0041	-64.4492	-50.5563	-2.1647	-2.2943
0.5261	0.3786	1340	0.5239	-0.7013	-1.6949	0.7116	0.9936	-64.4018	-50.5352	-2.1676	-2.2973
0.5825	0.3843	1360	0.5253	-0.7513	-1.7385	0.7153	0.9872	-64.5109	-50.6602	-2.1503	-2.2809
0.4382	0.3899	1380	0.5238	-0.8284	-1.8536	0.7265	1.0253	-64.7986	-50.8529	-2.1373	-2.2682
0.6441	0.3956	1400	0.5219	-0.7553	-1.7853	0.7153	1.0300	-64.6277	-50.6702	-2.1612	-2.2921
0.5047	0.4012	1420	0.5237	-0.6198	-1.6539	0.7116	1.0340	-64.2992	-50.3315	-2.1983	-2.3295
0.5307	0.4069	1440	0.5236	-0.6487	-1.6732	0.7067	1.0245	-64.3476	-50.4038	-2.1875	-2.3189
0.4699	0.4125	1460	0.5201	-0.7075	-1.7583	0.7153	1.0508	-64.5602	-50.5506	-2.1726	-2.3044
0.3169	0.4182	1480	0.5218	-0.7439	-1.8221	0.7277	1.0782	-64.7197	-50.6417	-2.1755	-2.3080
0.4566	0.4238	1500	0.5205	-0.7491	-1.7970	0.7042	1.0479	-64.6569	-50.6547	-2.1756	-2.3079
0.37	0.4295	1520	0.5211	-0.7699	-1.8319	0.7240	1.0620	-64.7442	-50.7067	-2.1687	-2.3010
0.4884	0.4352	1540	0.5205	-0.7736	-1.8278	0.7129	1.0541	-64.7340	-50.7161	-2.1712	-2.3041
0.4276	0.4408	1560	0.5229	-0.7547	-1.8057	0.7252	1.0510	-64.6788	-50.6687	-2.1745	-2.3077
0.4672	0.4465	1580	0.5218	-0.8091	-1.8726	0.7302	1.0634	-64.8460	-50.8048	-2.1630	-2.2966
0.5569	0.4521	1600	0.5215	-0.7958	-1.8601	0.7153	1.0643	-64.8148	-50.7715	-2.1721	-2.3050
0.4779	0.4578	1620	0.5207	-0.8273	-1.8951	0.7153	1.0677	-64.9022	-50.8503	-2.1695	-2.3018
0.4707	0.4634	1640	0.5206	-0.8558	-1.9396	0.7166	1.0838	-65.0135	-50.9215	-2.1745	-2.3078
0.3918	0.4691	1660	0.5221	-0.9415	-2.0410	0.7265	1.0996	-65.2671	-51.1356	-2.1480	-2.2808
0.6116	0.4747	1680	0.5234	-0.9811	-2.0746	0.7265	1.0935	-65.3511	-51.2348	-2.1321	-2.2650
0.3337	0.4804	1700	0.5220	-0.9777	-2.0761	0.7228	1.0985	-65.3549	-51.2262	-2.1379	-2.2718
0.4512	0.4860	1720	0.5206	-1.0201	-2.1376	0.7228	1.1175	-65.5085	-51.3322	-2.1410	-2.2750
0.582	0.4917	1740	0.5213	-0.9677	-2.0899	0.7228	1.1222	-65.3893	-51.2013	-2.1528	-2.2868
0.5398	0.4973	1760	0.5179	-0.8736	-1.9974	0.7290	1.1238	-65.1581	-50.9661	-2.1643	-2.2981
0.4398	0.5030	1780	0.5206	-0.9232	-2.0641	0.7314	1.1408	-65.3247	-51.0901	-2.1602	-2.2941
0.4233	0.5086	1800	0.5199	-0.9351	-2.0692	0.7302	1.1341	-65.3375	-51.1197	-2.1534	-2.2880
0.4632	0.5143	1820	0.5199	-0.9305	-2.0567	0.7302	1.1262	-65.3062	-51.1082	-2.1526	-2.2870
0.4585	0.5199	1840	0.5189	-0.8993	-2.0315	0.7265	1.1322	-65.2433	-51.0302	-2.1562	-2.2903
0.6299	0.5256	1860	0.5161	-0.9066	-2.0405	0.7178	1.1339	-65.2658	-51.0485	-2.1535	-2.2868
0.5437	0.5312	1880	0.5186	-0.8976	-2.0322	0.7277	1.1346	-65.2450	-51.0259	-2.1601	-2.2932
0.5668	0.5369	1900	0.5165	-0.9512	-2.1071	0.7351	1.1559	-65.4322	-51.1600	-2.1609	-2.2937
0.5985	0.5425	1920	0.5158	-1.0160	-2.1909	0.7265	1.1749	-65.6417	-51.3219	-2.1474	-2.2803
0.3948	0.5482	1940	0.5164	-0.9905	-2.1473	0.7290	1.1567	-65.5327	-51.2582	-2.1467	-2.2803
0.596	0.5538	1960	0.5169	-0.9997	-2.1681	0.7228	1.1684	-65.5848	-51.2813	-2.1461	-2.2799
0.423	0.5595	1980	0.5185	-1.0368	-2.2134	0.7203	1.1766	-65.6979	-51.3739	-2.1394	-2.2738
0.386	0.5651	2000	0.5163	-1.0438	-2.2222	0.7252	1.1784	-65.7199	-51.3915	-2.1471	-2.2814
0.4411	0.5708	2020	0.5177	-1.0343	-2.2240	0.7302	1.1897	-65.7246	-51.3677	-2.1558	-2.2911
0.7334	0.5764	2040	0.5171	-1.0332	-2.2105	0.7314	1.1773	-65.6907	-51.3649	-2.1478	-2.2828
0.5096	0.5821	2060	0.5164	-1.0285	-2.1999	0.7252	1.1714	-65.6643	-51.3533	-2.1379	-2.2719
0.5612	0.5877	2080	0.5177	-1.0767	-2.2596	0.7252	1.1829	-65.8136	-51.4737	-2.1338	-2.2684
0.6291	0.5934	2100	0.5172	-1.0758	-2.2597	0.7191	1.1839	-65.8138	-51.4715	-2.1426	-2.2777
0.5359	0.5990	2120	0.5152	-1.0621	-2.2517	0.7327	1.1896	-65.7938	-51.4373	-2.1440	-2.2788
0.3542	0.6047	2140	0.5155	-1.0848	-2.2786	0.7240	1.1938	-65.8610	-51.4940	-2.1410	-2.2765
0.4435	0.6103	2160	0.5149	-1.0458	-2.2384	0.7252	1.1926	-65.7605	-51.3965	-2.1456	-2.2803
0.6526	0.6160	2180	0.5145	-1.0393	-2.2256	0.7290	1.1863	-65.7285	-51.3802	-2.1432	-2.2782
0.6079	0.6216	2200	0.5147	-1.0593	-2.2419	0.7215	1.1826	-65.7693	-51.4302	-2.1347	-2.2695
0.4454	0.6273	2220	0.5150	-1.0606	-2.2458	0.7252	1.1852	-65.7790	-51.4335	-2.1361	-2.2712
0.4333	0.6329	2240	0.5129	-1.0212	-2.2077	0.7314	1.1865	-65.6838	-51.3350	-2.1436	-2.2782
0.4113	0.6386	2260	0.5136	-1.0312	-2.2084	0.7203	1.1772	-65.6856	-51.3599	-2.1463	-2.2815
0.6801	0.6442	2280	0.5131	-1.0369	-2.2164	0.7252	1.1796	-65.7056	-51.3741	-2.1432	-2.2775
0.5828	0.6499	2300	0.5131	-1.0410	-2.2116	0.7178	1.1706	-65.6935	-51.3844	-2.1373	-2.2714
0.5322	0.6556	2320	0.5134	-1.0123	-2.1817	0.7104	1.1694	-65.6188	-51.3127	-2.1462	-2.2807
0.5746	0.6612	2340	0.5110	-0.9878	-2.1669	0.7203	1.1791	-65.5817	-51.2514	-2.1484	-2.2827
0.4121	0.6669	2360	0.5118	-0.9939	-2.1757	0.7215	1.1818	-65.6038	-51.2667	-2.1499	-2.2845
0.4285	0.6725	2380	0.5120	-0.9965	-2.1797	0.7104	1.1832	-65.6137	-51.2732	-2.1509	-2.2853
0.4009	0.6782	2400	0.5124	-0.9943	-2.1806	0.7228	1.1863	-65.6160	-51.2677	-2.1482	-2.2832
0.3993	0.6838	2420	0.5131	-0.9900	-2.1671	0.7228	1.1771	-65.5822	-51.2569	-2.1479	-2.2825
0.5328	0.6895	2440	0.5142	-0.9856	-2.1574	0.7191	1.1719	-65.5581	-51.2459	-2.1550	-2.2904
0.3505	0.6951	2460	0.5132	-0.9556	-2.1299	0.7240	1.1743	-65.4893	-51.1710	-2.1580	-2.2930
0.3536	0.7008	2480	0.5127	-0.9248	-2.1049	0.7203	1.1801	-65.4267	-51.0939	-2.1719	-2.3073
0.3663	0.7064	2500	0.5105	-0.9315	-2.1153	0.7191	1.1838	-65.4528	-51.1106	-2.1706	-2.3063
0.5672	0.7121	2520	0.5106	-0.8970	-2.0766	0.7191	1.1796	-65.3560	-51.0244	-2.1762	-2.3118
0.4346	0.7177	2540	0.5113	-0.9022	-2.0805	0.7215	1.1782	-65.3657	-51.0376	-2.1727	-2.3080
0.5074	0.7234	2560	0.5095	-0.8990	-2.0813	0.7203	1.1823	-65.3679	-51.0296	-2.1789	-2.3147
0.4002	0.7290	2580	0.5089	-0.8912	-2.0816	0.7277	1.1904	-65.3685	-51.0099	-2.1811	-2.3166
0.5329	0.7347	2600	0.5102	-0.8853	-2.0742	0.7215	1.1889	-65.3500	-50.9952	-2.1828	-2.3189
0.3891	0.7403	2620	0.5106	-0.8639	-2.0527	0.7191	1.1888	-65.2962	-50.9416	-2.1872	-2.3233
0.3184	0.7460	2640	0.5092	-0.8743	-2.0582	0.7191	1.1839	-65.3099	-50.9677	-2.1832	-2.3190
0.6153	0.7516	2660	0.5073	-0.8925	-2.0840	0.7203	1.1915	-65.3746	-51.0133	-2.1753	-2.3103
0.5933	0.7573	2680	0.5067	-0.8906	-2.0888	0.7277	1.1982	-65.3866	-51.0085	-2.1752	-2.3107
0.4282	0.7629	2700	0.5075	-0.8878	-2.0790	0.7203	1.1912	-65.3620	-51.0015	-2.1807	-2.3166
0.496	0.7686	2720	0.5077	-0.8698	-2.0574	0.7240	1.1876	-65.3080	-50.9564	-2.1832	-2.3189
0.3643	0.7742	2740	0.5081	-0.8513	-2.0338	0.7252	1.1825	-65.2490	-50.9103	-2.1851	-2.3212
0.6116	0.7799	2760	0.5087	-0.8735	-2.0593	0.7252	1.1858	-65.3129	-50.9658	-2.1787	-2.3145
0.4386	0.7855	2780	0.5058	-0.8656	-2.0580	0.7215	1.1924	-65.3094	-50.9459	-2.1775	-2.3132
0.578	0.7912	2800	0.5085	-0.8637	-2.0508	0.7215	1.1871	-65.2916	-50.9413	-2.1811	-2.3169
0.6564	0.7968	2820	0.5088	-0.8694	-2.0609	0.7252	1.1915	-65.3168	-50.9555	-2.1791	-2.3148
0.3907	0.8025	2840	0.5084	-0.8685	-2.0504	0.7240	1.1819	-65.2906	-50.9533	-2.1793	-2.3150
0.3805	0.8081	2860	0.5070	-0.8648	-2.0510	0.7203	1.1862	-65.2919	-50.9440	-2.1766	-2.3125
0.4056	0.8138	2880	0.5078	-0.8703	-2.0588	0.7215	1.1886	-65.3116	-50.9576	-2.1760	-2.3112
0.5449	0.8194	2900	0.5083	-0.8718	-2.0628	0.7141	1.1910	-65.3216	-50.9616	-2.1809	-2.3169
0.6432	0.8251	2920	0.5087	-0.8672	-2.0709	0.7277	1.2038	-65.3418	-50.9499	-2.1845	-2.3208
0.487	0.8307	2940	0.5079	-0.8675	-2.0523	0.7252	1.1848	-65.2952	-50.9506	-2.1863	-2.3223
0.3892	0.8364	2960	0.5080	-0.8634	-2.0660	0.7290	1.2025	-65.3294	-50.9406	-2.1891	-2.3252
0.431	0.8420	2980	0.5082	-0.8836	-2.0755	0.7178	1.1919	-65.3532	-50.9910	-2.1809	-2.3165
0.3476	0.8477	3000	0.5094	-0.8806	-2.0603	0.7153	1.1797	-65.3152	-50.9834	-2.1825	-2.3186
0.3878	0.8533	3020	0.5070	-0.8775	-2.0680	0.7191	1.1905	-65.3346	-50.9758	-2.1817	-2.3176
0.4734	0.8590	3040	0.5071	-0.8777	-2.0675	0.7302	1.1898	-65.3332	-50.9763	-2.1805	-2.3163
0.5006	0.8647	3060	0.5079	-0.8768	-2.0809	0.7166	1.2041	-65.3667	-50.9740	-2.1838	-2.3203
0.3146	0.8703	3080	0.5076	-0.9002	-2.0840	0.7203	1.1838	-65.3745	-51.0324	-2.1787	-2.3147
0.4258	0.8760	3100	0.5073	-0.8952	-2.0984	0.7178	1.2031	-65.4105	-51.0201	-2.1837	-2.3209
0.5796	0.8816	3120	0.5083	-0.8963	-2.0857	0.7191	1.1894	-65.3788	-51.0227	-2.1765	-2.3125
0.4278	0.8873	3140	0.5087	-0.8890	-2.0983	0.7215	1.2093	-65.4102	-51.0044	-2.1797	-2.3162
0.3634	0.8929	3160	0.5081	-0.8994	-2.0881	0.7203	1.1888	-65.3849	-51.0304	-2.1812	-2.3171
0.3873	0.8986	3180	0.5068	-0.8922	-2.0905	0.7228	1.1983	-65.3907	-51.0125	-2.1791	-2.3154
0.3168	0.9042	3200	0.5065	-0.8917	-2.0805	0.7153	1.1888	-65.3658	-51.0112	-2.1820	-2.3185
0.5393	0.9099	3220	0.5080	-0.8953	-2.0888	0.7203	1.1936	-65.3866	-51.0201	-2.1804	-2.3168
0.4792	0.9155	3240	0.5082	-0.8911	-2.0829	0.7079	1.1917	-65.3717	-51.0098	-2.1820	-2.3183
0.5638	0.9212	3260	0.5067	-0.8959	-2.0989	0.7166	1.2030	-65.4119	-51.0217	-2.1788	-2.3147
0.5061	0.9268	3280	0.5069	-0.8997	-2.0862	0.7141	1.1865	-65.3801	-51.0312	-2.1758	-2.3114
0.3982	0.9325	3300	0.5084	-0.8987	-2.1019	0.7191	1.2032	-65.4194	-51.0287	-2.1793	-2.3157
0.3486	0.9381	3320	0.5074	-0.8962	-2.1095	0.7203	1.2133	-65.4384	-51.0225	-2.1775	-2.3139
0.4772	0.9438	3340	0.5087	-0.9011	-2.1037	0.7203	1.2025	-65.4237	-51.0348	-2.1766	-2.3128
0.3378	0.9494	3360	0.5082	-0.8969	-2.0914	0.7129	1.1945	-65.3932	-51.0243	-2.1738	-2.3095
0.3605	0.9551	3380	0.5057	-0.8998	-2.1158	0.7252	1.2160	-65.4541	-51.0315	-2.1784	-2.3147
0.3477	0.9607	3400	0.5079	-0.9038	-2.1048	0.7252	1.2009	-65.4264	-51.0416	-2.1762	-2.3121
0.6847	0.9664	3420	0.5083	-0.9062	-2.1018	0.7252	1.1956	-65.4190	-51.0474	-2.1810	-2.3174
0.3043	0.9720	3440	0.5098	-0.9032	-2.1082	0.7153	1.2050	-65.4350	-51.0399	-2.1781	-2.3144
0.6092	0.9777	3460	0.5074	-0.9075	-2.1063	0.7240	1.1988	-65.4302	-51.0507	-2.1763	-2.3124
0.4301	0.9833	3480	0.5063	-0.9102	-2.1158	0.7240	1.2056	-65.4541	-51.0575	-2.1757	-2.3115
0.3835	0.9890	3500	0.5075	-0.9011	-2.1013	0.7191	1.2002	-65.4177	-51.0347	-2.1760	-2.3119
0.4456	0.9946	3520	0.5059	-0.9026	-2.1124	0.7215	1.2098	-65.4456	-51.0386	-2.1791	-2.3155

Framework versions

Transformers 4.45.2
Pytorch 2.5.1+cu121
Datasets 3.5.0
Tokenizers 0.20.3

YuchenLi01/generatedSoftQwen2.5MathRM72Bth0.5MoreNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42

generatedSoftQwen2.5MathRM72Bth0.5MoreNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

YuchenLi01/
generatedSoftQwen2.5MathRM72Bth0.5MoreNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42