This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_DPO_MoreUniqueResponseNoGTv2 dataset. It achieves the following results on the evaluation set:

Loss: 0.5796
Rewards/chosen: -2.4031
Rewards/rejected: -4.5651
Rewards/accuracies: 0.7259
Rewards/margins: 2.1620
Logps/rejected: -68.2860
Logps/chosen: -50.5201
Logits/rejected: -2.1994
Logits/chosen: -2.3640

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Logits/chosen	Logits/rejected	Logps/chosen	Logps/rejected	Validation Loss	Rewards/accuracies	Rewards/chosen	Rewards/margins	Rewards/rejected
0.73	0.0060	20	-2.2840	-2.1666	-44.5255	-56.8519	0.6995	0.4933	0.0048	-0.0063	0.0111
0.6901	0.0120	40	-2.2842	-2.1669	-44.5031	-56.8548	0.7003	0.5120	0.0138	0.0039	0.0099
0.7075	0.0180	60	-2.2843	-2.1666	-44.5252	-56.9059	0.6981	0.5294	0.0050	0.0155	-0.0105
0.7391	0.0240	80	-2.2858	-2.1679	-44.5320	-56.8932	0.6966	0.4973	0.0022	0.0077	-0.0055
0.6997	0.0300	100	-2.2827	-2.1646	-44.5602	-56.9502	0.6936	0.5414	-0.0090	0.0192	-0.0283
0.6947	0.0360	120	-2.2828	-2.1639	-44.5720	-56.9823	0.6887	0.5561	-0.0138	0.0274	-0.0411
0.6603	0.0420	140	-2.2806	-2.1608	-44.6200	-57.0791	0.6812	0.5789	-0.0330	0.0468	-0.0798
0.6614	0.0480	160	-2.2676	-2.1469	-44.7167	-57.2574	0.6712	0.6283	-0.0717	0.0795	-0.1511
0.6172	0.0540	180	-2.2744	-2.1516	-44.7115	-57.3140	0.6609	0.6243	-0.0696	0.1042	-0.1738
0.6768	0.0600	200	-2.2685	-2.1434	-44.9507	-57.6549	0.6521	0.6150	-0.1653	0.1449	-0.3102
0.6434	0.0660	220	-2.2714	-2.1442	-44.8940	-57.7109	0.6400	0.6310	-0.1426	0.1900	-0.3325
0.6278	0.0720	240	-2.2784	-2.1486	-44.9654	-57.8901	0.6312	0.6364	-0.1711	0.2331	-0.4042
0.6406	0.0780	260	-2.2827	-2.1504	-44.9851	-58.0680	0.6199	0.6524	-0.1790	0.2963	-0.4754
0.5932	0.0840	280	-2.2845	-2.1501	-45.0792	-58.2691	0.6100	0.6644	-0.2167	0.3392	-0.5558
0.5691	0.0900	300	-2.2889	-2.1523	-45.0835	-58.4435	0.5995	0.6885	-0.2184	0.4072	-0.6256
0.632	0.0960	320	-2.2982	-2.1602	-45.0706	-58.5789	0.5903	0.6979	-0.2132	0.4665	-0.6797
0.4984	0.1019	340	-2.2795	-2.1403	-45.4087	-59.1380	0.5824	0.7112	-0.3485	0.5549	-0.9034
0.4785	0.1079	360	-2.2724	-2.1301	-45.7659	-59.7558	0.5792	0.6965	-0.4913	0.6592	-1.1505
0.4918	0.1139	380	-2.3079	-2.1635	-45.5944	-59.7881	0.5690	0.7126	-0.4227	0.7407	-1.1634
0.4795	0.1199	400	-2.3040	-2.1581	-45.6597	-59.9433	0.5674	0.7112	-0.4489	0.7767	-1.2255
0.4556	0.1259	420	-2.3335	-2.1853	-45.5978	-60.0391	0.5630	0.7246	-0.4241	0.8397	-1.2638
0.5614	0.1319	440	-2.3154	-2.1660	-45.9202	-60.5862	0.5540	0.7380	-0.5531	0.9296	-1.4827
0.5586	0.1379	460	-2.2845	-2.1360	-46.4138	-61.1683	0.5535	0.7193	-0.7505	0.9650	-1.7155
0.4678	0.1439	480	-2.2856	-2.1359	-46.5777	-61.5364	0.5485	0.7246	-0.8161	1.0467	-1.8627
0.4443	0.1499	500	-2.3041	-2.1535	-46.6548	-61.7966	0.5465	0.7326	-0.8469	1.1199	-1.9668
0.581	0.1559	520	-2.3238	-2.1717	-46.5970	-61.9235	0.5456	0.7447	-0.8238	1.1938	-2.0176
0.5663	0.1619	540	-2.3158	-2.1632	-46.7866	-62.1512	0.5429	0.7447	-0.8996	1.2091	-2.1087
0.4027	0.1679	560	-2.2722	-2.1216	-47.1982	-62.6455	0.5407	0.7380	-1.0642	1.2421	-2.3064
0.4467	0.1739	580	-2.3056	-2.1547	-46.8656	-62.2133	0.5352	0.7313	-0.9312	1.2023	-2.1335
0.5236	0.1799	600	-2.3359	-2.1846	-46.3751	-61.6073	0.5316	0.7353	-0.7350	1.1561	-1.8911
0.3381	0.1859	620	-2.3439	-2.1896	-46.7483	-62.2009	0.5381	0.7393	-0.8843	1.2443	-2.1285
0.4791	0.1919	640	-2.3372	-2.1839	-46.7165	-62.2024	0.5352	0.7313	-0.8716	1.2576	-2.1291
0.6989	0.1979	660	-2.3108	-2.1578	-47.0022	-62.4652	0.5357	0.7259	-0.9859	1.2484	-2.2343
0.4338	0.2039	680	-2.3471	-2.1925	-46.8417	-62.5038	0.5333	0.7313	-0.9216	1.3281	-2.2497
0.4499	0.2099	700	-2.3713	-2.2142	-46.9819	-62.6967	0.5393	0.7259	-0.9777	1.3491	-2.3269
0.3553	0.2159	720	-2.3600	-2.2035	-46.9457	-62.6362	0.5400	0.7273	-0.9633	1.3394	-2.3027
0.3781	0.2219	740	-2.3937	-2.2379	-46.8394	-62.6171	0.5436	0.7166	-0.9207	1.3743	-2.2950
0.4113	0.2279	760	-2.3351	-2.1804	-47.5026	-63.2906	0.5362	0.7259	-1.1860	1.3784	-2.5644
0.5919	0.2339	780	-2.2881	-2.1339	-48.1201	-64.1256	0.5331	0.7233	-1.4330	1.4654	-2.8984
0.4062	0.2399	800	-2.2835	-2.1289	-48.4004	-64.5414	0.5359	0.7299	-1.5452	1.5196	-3.0648
0.402	0.2459	820	0.5425	-1.4537	-2.9836	0.7219	1.5298	-64.3322	-48.1467	-2.1768	-2.3319
0.4196	0.2519	840	0.5325	-1.4218	-2.8913	0.7326	1.4695	-64.1016	-48.0669	-2.1561	-2.3107
0.4996	0.2579	860	0.5375	-1.3693	-2.8260	0.7219	1.4567	-63.9384	-47.9357	-2.1802	-2.3355
0.4782	0.2639	880	0.5361	-1.7809	-3.2423	0.7340	1.4614	-64.9790	-48.9646	-2.1078	-2.2600
0.4474	0.2699	900	0.5341	-1.9701	-3.5463	0.7353	1.5763	-65.7392	-49.4376	-2.0654	-2.2167
0.4448	0.2759	920	0.5391	-1.6563	-3.3247	0.7380	1.6683	-65.1850	-48.6532	-2.1765	-2.3326
0.5457	0.2819	940	0.5395	-1.8977	-3.6369	0.7420	1.7392	-65.9655	-49.2566	-2.1511	-2.3052
0.4932	0.2879	960	0.5398	-1.8585	-3.5454	0.7273	1.6868	-65.7367	-49.1587	-2.1405	-2.2936
0.4699	0.2939	980	0.5353	-1.7706	-3.4101	0.7313	1.6395	-65.3987	-48.9389	-2.1467	-2.2993
0.4109	0.2999	1000	0.5267	-1.5880	-3.1869	0.7313	1.5989	-64.8406	-48.4824	-2.1632	-2.3183
0.5067	0.3058	1020	0.5287	-1.6588	-3.2880	0.7460	1.6292	-65.0934	-48.6594	-2.1457	-2.2990
0.3888	0.3118	1040	0.5338	-1.8245	-3.5016	0.7366	1.6771	-65.6273	-49.0736	-2.1307	-2.2831
0.3982	0.3178	1060	0.5368	-1.9154	-3.6970	0.7286	1.7816	-66.1159	-49.3009	-2.1567	-2.3118
0.3106	0.3238	1080	0.5404	-2.0885	-3.8850	0.7286	1.7965	-66.5857	-49.7337	-2.1295	-2.2842
0.2842	0.3298	1100	0.5440	-2.1413	-3.9725	0.7393	1.8311	-66.8044	-49.8657	-2.1367	-2.2916
0.3494	0.3358	1120	0.5412	-1.9277	-3.6556	0.7246	1.7278	-66.0122	-49.3317	-2.1580	-2.3135
0.3945	0.3418	1140	0.5399	-1.9763	-3.7762	0.7259	1.7999	-66.3138	-49.4531	-2.1267	-2.2828
0.4955	0.3478	1160	0.5403	-1.9863	-3.7766	0.7433	1.7902	-66.3148	-49.4782	-2.1318	-2.2895
0.4077	0.3538	1180	0.5338	-1.9426	-3.6992	0.7460	1.7566	-66.1213	-49.3688	-2.1064	-2.2607
0.3905	0.3598	1200	0.5304	-1.8037	-3.5632	0.7433	1.7594	-65.7812	-49.0217	-2.1326	-2.2863
0.3377	0.3658	1220	0.5363	-1.8900	-3.6889	0.7380	1.7989	-66.0957	-49.2374	-2.1442	-2.2971
0.7295	0.3718	1240	0.5400	-2.0031	-3.8320	0.7326	1.8289	-66.4533	-49.5202	-2.1355	-2.2904
0.4121	0.3778	1260	0.5364	-2.1489	-3.9627	0.7326	1.8138	-66.7800	-49.8845	-2.0910	-2.2457
0.5229	0.3838	1280	0.5499	-2.3115	-4.1540	0.7353	1.8425	-67.2584	-50.2912	-2.0893	-2.2437
0.2398	0.3898	1300	0.5511	-2.0555	-3.9019	0.7166	1.8464	-66.6280	-49.6512	-2.1689	-2.3252
0.3229	0.3958	1320	0.5551	-2.2235	-4.1544	0.7206	1.9309	-67.2594	-50.0712	-2.1641	-2.3209
0.2727	0.4018	1340	0.5572	-2.1956	-4.1986	0.7206	2.0031	-67.3699	-50.0013	-2.1849	-2.3431
0.302	0.4078	1360	0.5551	-2.3439	-4.3123	0.7126	1.9684	-67.6541	-50.3721	-2.1351	-2.2933
0.4957	0.4138	1380	0.5526	-2.2890	-4.2712	0.7313	1.9822	-67.5514	-50.2349	-2.1510	-2.3100
0.2406	0.4198	1400	0.5539	-2.4001	-4.3914	0.7340	1.9913	-67.8518	-50.5127	-2.1305	-2.2879
0.5632	0.4258	1420	0.5470	-2.3739	-4.3738	0.7406	1.9998	-67.8077	-50.4472	-2.1245	-2.2824
0.5002	0.4318	1440	0.5428	-2.1356	-4.1302	0.7313	1.9946	-67.1989	-49.8514	-2.1741	-2.3345
0.4629	0.4378	1460	0.5534	-2.3573	-4.3582	0.7246	2.0009	-67.7688	-50.4055	-2.1541	-2.3134
0.4672	0.4438	1480	0.5538	-2.4371	-4.4549	0.7246	2.0178	-68.0106	-50.6052	-2.1433	-2.3020
0.4207	0.4498	1500	0.5538	-2.2938	-4.2942	0.7273	2.0004	-67.6089	-50.2469	-2.1491	-2.3077
0.4197	0.4558	1520	0.5567	-2.0589	-4.0578	0.7286	1.9990	-67.0179	-49.6596	-2.1905	-2.3514
0.2704	0.4618	1540	0.5672	-2.1422	-4.2660	0.7259	2.1238	-67.5383	-49.8678	-2.2180	-2.3816
0.4166	0.4678	1560	0.5728	-2.1945	-4.2757	0.7246	2.0812	-67.5626	-49.9987	-2.2067	-2.3695
0.4362	0.4738	1580	0.5635	-1.9547	-4.0054	0.7366	2.0506	-66.8867	-49.3992	-2.2373	-2.4015
0.4167	0.4798	1600	0.5654	-2.0804	-4.0920	0.7313	2.0116	-67.1033	-49.7134	-2.2181	-2.3810
0.2316	0.4858	1620	0.5662	-2.2136	-4.2342	0.7313	2.0206	-67.4589	-50.0464	-2.1831	-2.3451
0.4188	0.4918	1640	0.5704	-2.3708	-4.4024	0.7313	2.0316	-67.8794	-50.4394	-2.1658	-2.3257
0.4348	0.4978	1660	0.5822	-2.3493	-4.3898	0.7326	2.0406	-67.8479	-50.3855	-2.1985	-2.3583
0.3002	0.5037	1680	0.5765	-2.3441	-4.3592	0.7313	2.0151	-67.7712	-50.3726	-2.1771	-2.3380
0.3789	0.5097	1700	0.5690	-2.5372	-4.5090	0.7259	1.9718	-68.1458	-50.8554	-2.1230	-2.2824
0.2039	0.5157	1720	0.5634	-2.3945	-4.3457	0.7219	1.9512	-67.7375	-50.4986	-2.1400	-2.3002
0.2798	0.5217	1740	0.5700	-2.3848	-4.3055	0.7233	1.9207	-67.6372	-50.4744	-2.1556	-2.3162
0.5354	0.5277	1760	0.5749	-2.2577	-4.1709	0.7233	1.9131	-67.3005	-50.1567	-2.1842	-2.3437
0.2853	0.5337	1780	0.5726	-2.2228	-4.0961	0.7126	1.8733	-67.1136	-50.0695	-2.1802	-2.3398
0.4659	0.5397	1800	0.5801	-2.3238	-4.3205	0.7166	1.9967	-67.6745	-50.3218	-2.1899	-2.3521
0.3181	0.5457	1820	0.5868	-2.3172	-4.3619	0.7139	2.0447	-67.7779	-50.3053	-2.2033	-2.3668
0.2995	0.5517	1840	0.5960	-2.3013	-4.3722	0.7112	2.0709	-67.8039	-50.2656	-2.2338	-2.3981
0.2868	0.5577	1860	0.5885	-2.2752	-4.3480	0.7273	2.0728	-67.7434	-50.2004	-2.2372	-2.4012
0.4067	0.5637	1880	0.5886	-2.4603	-4.5539	0.7353	2.0936	-68.2581	-50.6632	-2.2037	-2.3668
0.3498	0.5697	1900	0.5938	-2.5665	-4.6559	0.7353	2.0894	-68.5130	-50.9286	-2.1927	-2.3554
0.4811	0.5757	1920	0.5909	-2.4908	-4.5922	0.7299	2.1014	-68.3537	-50.7393	-2.2007	-2.3638
0.4587	0.5817	1940	0.5777	-2.4542	-4.5137	0.7353	2.0595	-68.1575	-50.6478	-2.1851	-2.3468
0.4067	0.5877	1960	0.5717	-2.3636	-4.4128	0.7259	2.0492	-67.9054	-50.4213	-2.2004	-2.3618
0.4554	0.5937	1980	0.5801	-2.3060	-4.3300	0.7233	2.0240	-67.6984	-50.2774	-2.2125	-2.3744
0.2715	0.5997	2000	0.5715	-2.1497	-4.1001	0.7353	1.9504	-67.1237	-49.8867	-2.2217	-2.3834
0.2609	0.6057	2020	0.5718	-2.1471	-4.0842	0.7353	1.9371	-67.0837	-49.8801	-2.2209	-2.3818
0.4077	0.6117	2040	0.5799	-2.1552	-4.1169	0.7219	1.9618	-67.1656	-49.9003	-2.2297	-2.3910
0.4469	0.6177	2060	0.5849	-2.3088	-4.3069	0.7206	1.9981	-67.6406	-50.2844	-2.2063	-2.3674
0.2778	0.6237	2080	0.5918	-2.4319	-4.4343	0.7193	2.0024	-67.9590	-50.5921	-2.1896	-2.3504
0.372	0.6297	2100	0.5929	-2.3451	-4.3485	0.7193	2.0034	-67.7445	-50.3751	-2.2062	-2.3680
0.422	0.6357	2120	0.5910	-2.3439	-4.3031	0.7139	1.9592	-67.6311	-50.3722	-2.2004	-2.3618
0.3289	0.6417	2140	0.5902	-2.4222	-4.3934	0.7206	1.9711	-67.8568	-50.5680	-2.1866	-2.3477
0.1748	0.6477	2160	0.5948	-2.4349	-4.4266	0.7139	1.9917	-67.9398	-50.5997	-2.1947	-2.3564
0.1898	0.6537	2180	0.5905	-2.4372	-4.4682	0.7233	2.0310	-68.0439	-50.6054	-2.1951	-2.3574
0.5178	0.6597	2200	0.5858	-2.4300	-4.4922	0.7193	2.0622	-68.1039	-50.5875	-2.1955	-2.3577
0.3765	0.6657	2220	0.5812	-2.3328	-4.3769	0.7233	2.0442	-67.8157	-50.3444	-2.2030	-2.3650
0.3267	0.6717	2240	0.5845	-2.3069	-4.3729	0.7206	2.0660	-67.8056	-50.2797	-2.2201	-2.3831
0.45	0.6777	2260	0.5832	-2.3386	-4.4083	0.7259	2.0697	-67.8940	-50.3588	-2.2113	-2.3739
0.1942	0.6837	2280	0.5841	-2.3967	-4.4887	0.7219	2.0919	-68.0950	-50.5043	-2.2112	-2.3739
0.2168	0.6897	2300	0.5862	-2.4025	-4.5129	0.7126	2.1104	-68.1556	-50.5187	-2.2138	-2.3776
0.2433	0.6957	2320	0.5842	-2.4185	-4.5170	0.7193	2.0985	-68.1658	-50.5587	-2.1972	-2.3602
0.6285	0.7016	2340	0.5835	-2.3343	-4.4543	0.7099	2.1200	-68.0092	-50.3482	-2.2223	-2.3854
0.3142	0.7076	2360	0.5830	-2.3351	-4.4725	0.7219	2.1374	-68.0546	-50.3502	-2.2216	-2.3850
0.2793	0.7136	2380	0.5810	-2.2839	-4.4001	0.7193	2.1162	-67.8736	-50.2222	-2.2298	-2.3934
0.4477	0.7196	2400	0.5811	-2.3245	-4.4509	0.7233	2.1264	-68.0005	-50.3235	-2.2198	-2.3831
0.6407	0.7256	2420	0.5835	-2.4206	-4.5564	0.7219	2.1358	-68.2643	-50.5640	-2.2129	-2.3757
0.6725	0.7316	2440	0.5829	-2.3928	-4.5300	0.7219	2.1372	-68.1984	-50.4945	-2.2144	-2.3779
0.1539	0.7376	2460	0.5824	-2.3620	-4.4853	0.7086	2.1233	-68.0866	-50.4174	-2.2106	-2.3743
0.5217	0.7436	2480	0.5806	-2.3462	-4.4794	0.7152	2.1332	-68.0717	-50.3778	-2.2045	-2.3682
0.5927	0.7496	2500	0.5792	-2.3323	-4.4373	0.7206	2.1050	-67.9665	-50.3432	-2.2054	-2.3687
0.2391	0.7556	2520	0.5779	-2.3913	-4.4956	0.7139	2.1043	-68.1123	-50.4905	-2.1976	-2.3612
0.3963	0.7616	2540	0.5780	-2.3785	-4.4885	0.7139	2.1100	-68.0947	-50.4587	-2.1989	-2.3626
0.3865	0.7676	2560	0.5771	-2.3431	-4.4701	0.7233	2.1269	-68.0485	-50.3702	-2.1976	-2.3612
0.3115	0.7736	2580	0.5771	-2.3741	-4.4909	0.7166	2.1168	-68.1005	-50.4475	-2.1920	-2.3557
0.1457	0.7796	2600	0.5763	-2.3594	-4.4667	0.7166	2.1073	-68.0401	-50.4109	-2.1976	-2.3614
0.4248	0.7856	2620	0.5761	-2.3756	-4.4871	0.7099	2.1115	-68.0910	-50.4513	-2.1979	-2.3620
0.2367	0.7916	2640	0.5792	-2.3799	-4.5087	0.7206	2.1288	-68.1450	-50.4620	-2.2027	-2.3667
0.2425	0.7976	2660	0.5778	-2.3999	-4.5289	0.7233	2.1290	-68.1955	-50.5122	-2.2006	-2.3649
0.2228	0.8036	2680	0.5791	-2.4102	-4.5374	0.7193	2.1272	-68.2169	-50.5379	-2.1994	-2.3633
0.3514	0.8096	2700	0.5757	-2.3855	-4.5265	0.7246	2.1410	-68.1895	-50.4760	-2.1996	-2.3628
0.2107	0.8156	2720	0.5787	-2.3971	-4.5519	0.7166	2.1548	-68.2531	-50.5052	-2.2035	-2.3671
0.2919	0.8216	2740	0.5821	-2.4521	-4.6157	0.7259	2.1636	-68.4126	-50.6426	-2.2049	-2.3685
0.1872	0.8276	2760	0.5830	-2.4473	-4.6201	0.7206	2.1728	-68.4235	-50.6306	-2.1994	-2.3626
0.6285	0.8336	2780	0.5796	-2.4423	-4.6060	0.7259	2.1637	-68.3884	-50.6182	-2.2029	-2.3668
0.4219	0.8396	2800	0.5805	-2.4320	-4.5906	0.7259	2.1586	-68.3497	-50.5923	-2.1985	-2.3620
0.2696	0.8456	2820	0.5803	-2.4421	-4.5977	0.7273	2.1557	-68.3676	-50.6175	-2.1955	-2.3590
0.2871	0.8516	2840	0.5802	-2.4455	-4.5991	0.7152	2.1536	-68.3710	-50.6261	-2.2014	-2.3651
0.4357	0.8576	2860	0.5799	-2.4497	-4.6074	0.7219	2.1577	-68.3918	-50.6366	-2.2018	-2.3657
0.3964	0.8636	2880	0.5788	-2.4219	-4.5952	0.7273	2.1732	-68.3613	-50.5672	-2.2017	-2.3658
0.2754	0.8696	2900	0.5779	-2.4233	-4.5869	0.7233	2.1636	-68.3405	-50.5706	-2.1984	-2.3626
0.2423	0.8756	2920	0.5776	-2.4189	-4.5939	0.7326	2.1750	-68.3581	-50.5597	-2.2030	-2.3674
0.2489	0.8816	2940	0.5801	-2.4346	-4.6080	0.7152	2.1734	-68.3933	-50.5989	-2.2014	-2.3658
0.2686	0.8876	2960	0.5837	-2.4470	-4.6246	0.7219	2.1776	-68.4349	-50.6298	-2.2037	-2.3685
0.3056	0.8936	2980	0.5801	-2.4300	-4.6104	0.7219	2.1804	-68.3993	-50.5873	-2.2055	-2.3700
0.3823	0.8996	3000	0.5832	-2.4193	-4.5819	0.7219	2.1626	-68.3280	-50.5606	-2.2080	-2.3728
0.4871	0.9055	3020	0.5810	-2.4152	-4.5746	0.7193	2.1593	-68.3097	-50.5504	-2.2041	-2.3687
0.2968	0.9115	3040	0.5803	-2.4074	-4.5805	0.7273	2.1731	-68.3245	-50.5308	-2.2070	-2.3717
0.3973	0.9175	3060	0.5816	-2.3949	-4.5655	0.7233	2.1706	-68.2870	-50.4997	-2.2066	-2.3713
0.2556	0.9235	3080	0.5794	-2.3923	-4.5685	0.7206	2.1761	-68.2945	-50.4932	-2.2024	-2.3669
0.3109	0.9295	3100	0.5789	-2.4031	-4.5590	0.7219	2.1559	-68.2708	-50.5201	-2.2030	-2.3676
0.2311	0.9355	3120	0.5767	-2.3930	-4.5679	0.7206	2.1749	-68.2930	-50.4949	-2.2025	-2.3673
0.2843	0.9415	3140	0.5809	-2.4034	-4.5747	0.7179	2.1714	-68.3102	-50.5208	-2.1989	-2.3632
0.2231	0.9475	3160	0.5802	-2.4029	-4.5586	0.7179	2.1557	-68.2699	-50.5196	-2.2057	-2.3706
0.3034	0.9535	3180	0.5797	-2.4012	-4.5643	0.7219	2.1631	-68.2840	-50.5154	-2.2038	-2.3686
0.325	0.9595	3200	0.5793	-2.4034	-4.5657	0.7193	2.1623	-68.2876	-50.5208	-2.2011	-2.3656
0.1966	0.9655	3220	0.5807	-2.4164	-4.5797	0.7112	2.1633	-68.3227	-50.5535	-2.2024	-2.3669
0.2471	0.9715	3240	0.5760	-2.3994	-4.5701	0.7206	2.1707	-68.2985	-50.5108	-2.1998	-2.3643
0.3629	0.9775	3260	0.5796	-2.4081	-4.5658	0.7273	2.1576	-68.2877	-50.5327	-2.2013	-2.3659
0.2003	0.9835	3280	0.5789	-2.4072	-4.5666	0.7219	2.1594	-68.2899	-50.5304	-2.2001	-2.3647
0.3775	0.9895	3300	0.5808	-2.4042	-4.5693	0.7139	2.1651	-68.2966	-50.5228	-2.1977	-2.3621
0.4797	0.9955	3320	0.5773	-2.3883	-4.5725	0.7193	2.1843	-68.3047	-50.4831	-2.1992	-2.3638

Framework versions

Transformers 4.45.2
Pytorch 2.5.1+cu121
Datasets 3.5.0
Tokenizers 0.20.3

YuchenLi01/generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.4_42

generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.4_42

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

YuchenLi01/
generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.4_42