This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_DPO_MoreUniqueResponseNoGTv2 dataset. It achieves the following results on the evaluation set:

Loss: 0.5551
Rewards/chosen: -3.0966
Rewards/rejected: -5.2547
Rewards/accuracies: 0.7340
Rewards/margins: 2.1582
Logps/rejected: -109.4268
Logps/chosen: -75.5033
Logits/rejected: -2.6367
Logits/chosen: -2.8370

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6986	0.0060	20	0.6939	0.0034	0.0000	0.5227	0.0033	-56.8792	-44.5040	-2.1649	-2.2823
0.6977	0.0120	40	0.6936	0.0029	0.0002	0.5361	0.0027	-56.8780	-44.5085	-2.1664	-2.2838
0.695	0.0180	60	0.6933	0.0004	-0.0003	0.5201	0.0008	-56.8829	-44.5331	-2.1690	-2.2866
0.7029	0.0240	80	0.6934	0.0022	-0.0012	0.5174	0.0034	-56.8914	-44.5153	-2.1643	-2.2821
0.6897	0.0300	100	0.6922	-0.0012	-0.0031	0.5160	0.0019	-56.9110	-44.5497	-2.1640	-2.2822
0.6961	0.0360	120	0.6905	-0.0038	-0.0140	0.5829	0.0102	-57.0194	-44.5755	-2.1634	-2.2824
0.6806	0.0420	140	0.6885	-0.0092	-0.0244	0.5869	0.0151	-57.1232	-44.6298	-2.1599	-2.2799
0.685	0.0480	160	0.6846	-0.0226	-0.0440	0.6136	0.0214	-57.3197	-44.7634	-2.1483	-2.2694
0.663	0.0540	180	0.6811	-0.0333	-0.0626	0.6136	0.0294	-57.5060	-44.8701	-2.1445	-2.2680
0.682	0.0600	200	0.6745	-0.0833	-0.1283	0.6029	0.0451	-58.1630	-45.3701	-2.1163	-2.2420
0.6776	0.0660	220	0.6664	-0.1152	-0.1820	0.6163	0.0668	-58.6994	-45.6896	-2.1019	-2.2309
0.6637	0.0720	240	0.6587	-0.1823	-0.2692	0.6016	0.0869	-59.5714	-46.3608	-2.0868	-2.2196
0.6753	0.0780	260	0.6490	-0.2435	-0.3693	0.6136	0.1258	-60.5723	-46.9727	-2.0639	-2.2014
0.6021	0.0840	280	0.6368	-0.3250	-0.4917	0.6377	0.1667	-61.7968	-47.7876	-2.0347	-2.1760
0.6268	0.0900	300	0.6224	-0.3840	-0.6041	0.6711	0.2201	-62.9202	-48.3775	-2.0403	-2.1866
0.6442	0.0960	320	0.6042	-0.4298	-0.7180	0.7045	0.2882	-64.0595	-48.8357	-2.0531	-2.2041
0.5183	0.1019	340	0.5888	-0.5615	-0.9369	0.7072	0.3754	-66.2488	-50.1528	-2.0371	-2.1917
0.5643	0.1079	360	0.5679	-0.8212	-1.3512	0.7179	0.5301	-70.3918	-52.7492	-2.0225	-2.1837
0.5271	0.1139	380	0.5530	-0.9505	-1.5941	0.7313	0.6436	-72.8206	-54.0429	-2.0849	-2.2512
0.4554	0.1199	400	0.5427	-0.9781	-1.7132	0.7460	0.7351	-74.0119	-54.3187	-2.1442	-2.3171
0.4057	0.1259	420	0.5347	-1.0445	-1.8458	0.7366	0.8013	-75.3372	-54.9825	-2.2197	-2.3976
0.5075	0.1319	440	0.5260	-1.1292	-2.0048	0.7353	0.8756	-76.9276	-55.8295	-2.2500	-2.4315
0.4946	0.1379	460	0.5205	-1.3054	-2.2641	0.7326	0.9587	-79.5203	-57.5918	-2.2241	-2.4065
0.4653	0.1439	480	0.5161	-1.3625	-2.3848	0.7340	1.0223	-80.7273	-58.1628	-2.2502	-2.4347
0.435	0.1499	500	0.5128	-1.3950	-2.4567	0.7353	1.0618	-81.4468	-58.4871	-2.3028	-2.4889
0.5137	0.1559	520	0.5102	-1.5070	-2.6596	0.7326	1.1526	-83.4755	-59.6072	-2.3774	-2.5667
0.465	0.1619	540	0.5039	-1.3675	-2.5003	0.7340	1.1328	-81.8828	-58.2127	-2.3771	-2.5651
0.3977	0.1679	560	0.5020	-1.4855	-2.6617	0.7326	1.1762	-83.4963	-59.3928	-2.3039	-2.4909
0.4281	0.1739	580	0.4991	-1.4210	-2.5670	0.7326	1.1460	-82.5495	-58.7472	-2.3028	-2.4890
0.425	0.1799	600	0.4969	-1.3829	-2.5062	0.7366	1.1233	-81.9415	-58.3668	-2.3374	-2.5236
0.3404	0.1859	620	0.4962	-1.4225	-2.5855	0.7340	1.1630	-82.7349	-58.7627	-2.3666	-2.5554
0.4214	0.1919	640	0.4970	-1.5379	-2.7749	0.7340	1.2370	-84.6286	-59.9166	-2.4263	-2.6170
0.5926	0.1979	660	0.4972	-1.6684	-2.9457	0.7406	1.2773	-86.3369	-61.2215	-2.4233	-2.6151
0.3889	0.2039	680	0.4916	-1.5946	-2.8390	0.7286	1.2444	-85.2694	-60.4837	-2.4156	-2.6061
0.4104	0.2099	700	0.4916	-1.6525	-2.9406	0.7366	1.2881	-86.2851	-61.0626	-2.4597	-2.6500
0.4041	0.2159	720	0.4921	-1.6918	-3.0140	0.7433	1.3221	-87.0193	-61.4558	-2.4812	-2.6704
0.3114	0.2219	740	0.4922	-1.7285	-3.0836	0.7340	1.3551	-87.7153	-61.8228	-2.5013	-2.6891
0.3639	0.2279	760	0.4925	-1.7037	-3.0533	0.7366	1.3496	-87.4126	-61.5744	-2.4938	-2.6821
0.4521	0.2339	780	0.4907	-1.7568	-3.1390	0.7326	1.3822	-88.2697	-62.1058	-2.4544	-2.6450
0.4358	0.2399	800	0.4906	-1.8896	-3.3322	0.7353	1.4426	-90.2014	-63.4332	-2.4341	-2.6261
0.3708	0.2459	820	0.4893	-1.9964	-3.4660	0.7353	1.4696	-91.5399	-64.5020	-2.4589	-2.6497
0.3818	0.2519	840	0.4862	-1.8270	-3.1945	0.7366	1.3675	-88.8247	-62.8074	-2.4101	-2.6001
0.4779	0.2579	860	0.4864	-1.8429	-3.2333	0.7393	1.3904	-89.2128	-62.9665	-2.4527	-2.6439
0.3753	0.2639	880	0.4898	-2.0843	-3.5310	0.7326	1.4467	-92.1892	-65.3801	-2.4187	-2.6103
0.4017	0.2699	900	0.4894	-2.1174	-3.6228	0.7433	1.5054	-93.1073	-65.7116	-2.3983	-2.5914
0.4991	0.2759	920	0.4898	-2.0142	-3.5459	0.7380	1.5316	-92.3381	-64.6800	-2.4743	-2.6689
0.396	0.2819	940	0.4884	-2.0193	-3.6092	0.7406	1.5899	-92.9719	-64.7305	-2.4929	-2.6868
0.4374	0.2879	960	0.4929	-2.1461	-3.7470	0.7259	1.6009	-94.3494	-65.9988	-2.5143	-2.7062
0.3937	0.2939	980	0.4907	-2.0457	-3.5884	0.7326	1.5428	-92.7638	-64.9943	-2.5037	-2.6955
0.3468	0.2999	1000	0.4821	-1.8902	-3.4236	0.7393	1.5334	-91.1151	-63.4391	-2.5042	-2.6979
0.387	0.3058	1020	0.4844	-1.9841	-3.5795	0.7393	1.5955	-92.6750	-64.3784	-2.5094	-2.7027
0.2359	0.3118	1040	0.4918	-2.2645	-3.9739	0.7393	1.7095	-96.6190	-67.1822	-2.5207	-2.7147
0.279	0.3178	1060	0.4889	-2.2684	-3.9975	0.7420	1.7291	-96.8550	-67.2216	-2.5559	-2.7513
0.3633	0.3238	1080	0.4909	-2.4194	-4.1798	0.7380	1.7603	-98.6773	-68.7320	-2.5212	-2.7176
0.268	0.3298	1100	0.4985	-2.6070	-4.4235	0.7340	1.8165	-101.1147	-70.6076	-2.4964	-2.6928
0.3377	0.3358	1120	0.4926	-2.3588	-4.1356	0.7406	1.7768	-98.2355	-68.1255	-2.5253	-2.7225
0.3561	0.3418	1140	0.4931	-2.3752	-4.1642	0.7460	1.7890	-98.5220	-68.2900	-2.5168	-2.7154
0.336	0.3478	1160	0.4932	-2.4544	-4.2492	0.7393	1.7948	-99.3716	-69.0816	-2.5201	-2.7191
0.4522	0.3538	1180	0.4933	-2.4253	-4.1992	0.7340	1.7739	-98.8715	-68.7902	-2.5250	-2.7220
0.3557	0.3598	1200	0.4909	-2.4608	-4.2346	0.7433	1.7738	-99.2255	-69.1459	-2.5185	-2.7138
0.3335	0.3658	1220	0.4922	-2.3184	-4.0631	0.7366	1.7447	-97.5100	-67.7212	-2.5595	-2.7529
0.4421	0.3718	1240	0.4930	-2.2029	-3.9398	0.7340	1.7369	-96.2771	-66.5666	-2.5602	-2.7550
0.3642	0.3778	1260	0.5048	-2.4578	-4.2587	0.7259	1.8009	-99.4665	-69.1156	-2.5327	-2.7295
0.4209	0.3838	1280	0.5197	-2.7500	-4.5938	0.7299	1.8438	-102.8178	-72.0374	-2.5185	-2.7159
0.249	0.3898	1300	0.5190	-2.7488	-4.5834	0.7326	1.8346	-102.7137	-72.0255	-2.5250	-2.7210
0.2509	0.3958	1320	0.5143	-2.6840	-4.5144	0.7233	1.8303	-102.0232	-71.3779	-2.5536	-2.7509
0.2123	0.4018	1340	0.5147	-2.6491	-4.4887	0.7246	1.8396	-101.7668	-71.0289	-2.6107	-2.8072
0.2796	0.4078	1360	0.5188	-2.5675	-4.3852	0.7233	1.8177	-100.7318	-70.2130	-2.5809	-2.7772
0.2664	0.4138	1380	0.5109	-2.4979	-4.3060	0.7286	1.8081	-99.9394	-69.5166	-2.5716	-2.7694
0.2358	0.4198	1400	0.5140	-2.7144	-4.6021	0.7273	1.8877	-102.9007	-71.6820	-2.5361	-2.7340
0.4291	0.4258	1420	0.5091	-2.6629	-4.6283	0.7353	1.9654	-103.1624	-71.1666	-2.5253	-2.7242
0.446	0.4318	1440	0.5115	-2.6574	-4.5887	0.7380	1.9312	-102.7662	-71.1118	-2.5296	-2.7290
0.3369	0.4378	1460	0.5247	-2.8253	-4.7631	0.7246	1.9378	-104.5102	-72.7903	-2.5197	-2.7189
0.4379	0.4438	1480	0.5289	-2.8241	-4.7772	0.7233	1.9531	-104.6512	-72.7784	-2.5021	-2.7020
0.3417	0.4498	1500	0.5073	-2.5268	-4.4450	0.7353	1.9182	-101.3300	-69.8056	-2.4740	-2.6718
0.4037	0.4558	1520	0.5062	-2.5825	-4.5083	0.7299	1.9259	-101.9630	-70.3623	-2.4768	-2.6746
0.2171	0.4618	1540	0.5135	-2.6433	-4.6261	0.7313	1.9828	-103.1403	-70.9701	-2.5756	-2.7767
0.2624	0.4678	1560	0.5286	-2.7761	-4.8000	0.7313	2.0239	-104.8792	-72.2985	-2.6357	-2.8371
0.4045	0.4738	1580	0.5222	-2.6559	-4.6640	0.7326	2.0081	-103.5199	-71.0967	-2.6607	-2.8615
0.294	0.4798	1600	0.5133	-2.5941	-4.5458	0.7273	1.9517	-102.3375	-70.4785	-2.6621	-2.8623
0.1813	0.4858	1620	0.5184	-2.7171	-4.6917	0.7259	1.9746	-103.7967	-71.7088	-2.6402	-2.8413
0.2311	0.4918	1640	0.5256	-2.7797	-4.7924	0.7246	2.0127	-104.8037	-72.3346	-2.6256	-2.8260
0.3562	0.4978	1660	0.5233	-2.7148	-4.7190	0.7340	2.0043	-104.0699	-71.6854	-2.5776	-2.7770
0.2517	0.5037	1680	0.5272	-2.7567	-4.7479	0.7326	1.9912	-104.3590	-72.1050	-2.5422	-2.7421
0.3723	0.5097	1700	0.5293	-2.8733	-4.8375	0.7299	1.9642	-105.2547	-73.2709	-2.5216	-2.7208
0.2091	0.5157	1720	0.5319	-2.7976	-4.7111	0.7299	1.9135	-103.9906	-72.5134	-2.5266	-2.7256
0.1921	0.5217	1740	0.5271	-2.7422	-4.5930	0.7206	1.8507	-102.8091	-71.9600	-2.4965	-2.6944
0.4062	0.5277	1760	0.5270	-2.6699	-4.5119	0.7219	1.8420	-101.9986	-71.2365	-2.5298	-2.7278
0.2481	0.5337	1780	0.5224	-2.5608	-4.3993	0.7206	1.8385	-100.8728	-70.1460	-2.5543	-2.7524
0.3967	0.5397	1800	0.5338	-2.7332	-4.6639	0.7219	1.9307	-103.5187	-71.8694	-2.5938	-2.7955
0.3565	0.5457	1820	0.5476	-2.9137	-4.9150	0.7299	2.0013	-106.0296	-73.6745	-2.6353	-2.8385
0.2046	0.5517	1840	0.5571	-2.9279	-4.9789	0.7233	2.0510	-106.6685	-73.8170	-2.7125	-2.9158
0.3236	0.5577	1860	0.5519	-2.8524	-4.9074	0.7273	2.0549	-105.9532	-73.0618	-2.7253	-2.9284
0.2967	0.5637	1880	0.5488	-2.8908	-4.9402	0.7259	2.0494	-106.2812	-73.4454	-2.6870	-2.8914
0.2968	0.5697	1900	0.5494	-2.9367	-4.9717	0.7246	2.0350	-106.5961	-73.9046	-2.6357	-2.8396
0.3967	0.5757	1920	0.5346	-2.7710	-4.7973	0.7273	2.0264	-104.8530	-72.2474	-2.5988	-2.8016
0.2982	0.5817	1940	0.5325	-2.8242	-4.8389	0.7286	2.0147	-105.2685	-72.7791	-2.5905	-2.7927
0.313	0.5877	1960	0.5403	-2.8877	-4.8776	0.7233	1.9899	-105.6555	-73.4145	-2.6110	-2.8127
0.3908	0.5937	1980	0.5457	-2.8524	-4.8182	0.7139	1.9658	-105.0613	-73.0615	-2.6133	-2.8143
0.2152	0.5997	2000	0.5339	-2.6962	-4.6113	0.7166	1.9151	-102.9926	-71.5001	-2.5983	-2.7985
0.2908	0.6057	2020	0.5329	-2.7064	-4.6428	0.7233	1.9364	-103.3080	-71.6017	-2.6209	-2.8204
0.3101	0.6117	2040	0.5447	-2.8242	-4.8085	0.7286	1.9843	-104.9641	-72.7791	-2.6757	-2.8753
0.4429	0.6177	2060	0.5486	-2.8973	-4.9057	0.7273	2.0084	-105.9366	-73.5108	-2.6928	-2.8924
0.2867	0.6237	2080	0.5525	-2.9741	-4.9854	0.7246	2.0113	-106.7334	-74.2783	-2.6726	-2.8725
0.2096	0.6297	2100	0.5532	-2.9958	-4.9891	0.7246	1.9933	-106.7708	-74.4960	-2.6714	-2.8706
0.3644	0.6357	2120	0.5503	-2.9607	-4.9074	0.7273	1.9467	-105.9533	-74.1447	-2.6479	-2.8466
0.2595	0.6417	2140	0.5485	-2.9529	-4.9122	0.7233	1.9593	-106.0015	-74.0661	-2.6366	-2.8343
0.1159	0.6477	2160	0.5471	-2.8491	-4.8549	0.7366	2.0059	-105.4288	-73.0282	-2.6552	-2.8536
0.1584	0.6537	2180	0.5499	-2.9723	-4.9878	0.7286	2.0155	-106.7574	-74.2602	-2.6517	-2.8500
0.4392	0.6597	2200	0.5464	-3.0274	-5.0990	0.7326	2.0716	-107.8697	-74.8114	-2.6535	-2.8522
0.373	0.6657	2220	0.5451	-3.0270	-5.0917	0.7299	2.0647	-107.7965	-74.8071	-2.6611	-2.8594
0.3986	0.6717	2240	0.5456	-2.9663	-4.9945	0.7286	2.0281	-106.8242	-74.2008	-2.6601	-2.8579
0.2906	0.6777	2260	0.5437	-2.9611	-4.9810	0.7219	2.0200	-106.6898	-74.1482	-2.6676	-2.8654
0.1618	0.6837	2280	0.5434	-2.9904	-5.0364	0.7326	2.0461	-107.2439	-74.4412	-2.6593	-2.8573
0.1724	0.6897	2300	0.5453	-3.0184	-5.0874	0.7299	2.0691	-107.7537	-74.7211	-2.6608	-2.8593
0.1537	0.6957	2320	0.5441	-2.9683	-5.0343	0.7246	2.0660	-107.2228	-74.2208	-2.6583	-2.8568
0.437	0.7016	2340	0.5465	-2.9922	-5.0747	0.7246	2.0825	-107.6264	-74.4599	-2.6706	-2.8688
0.2334	0.7076	2360	0.5510	-3.0577	-5.1554	0.7219	2.0978	-108.4339	-75.1141	-2.6790	-2.8778
0.1311	0.7136	2380	0.5519	-3.0657	-5.1583	0.7233	2.0926	-108.4627	-75.1945	-2.6839	-2.8826
0.4685	0.7196	2400	0.5543	-3.0670	-5.1705	0.7206	2.1035	-108.5843	-75.2077	-2.6764	-2.8747
0.4398	0.7256	2420	0.5546	-3.1039	-5.2026	0.7313	2.0988	-108.9060	-75.5765	-2.6590	-2.8579
0.3422	0.7316	2440	0.5527	-3.0644	-5.1622	0.7273	2.0978	-108.5015	-75.1817	-2.6663	-2.8654
0.1184	0.7376	2460	0.5517	-3.0498	-5.1575	0.7286	2.1077	-108.4545	-75.0353	-2.6608	-2.8595
0.3125	0.7436	2480	0.5517	-3.0349	-5.1217	0.7259	2.0867	-108.0961	-74.8868	-2.6481	-2.8475
0.5052	0.7496	2500	0.5532	-3.0783	-5.1719	0.7313	2.0936	-108.5982	-75.3203	-2.6471	-2.8466
0.1411	0.7556	2520	0.5567	-3.1577	-5.2483	0.7273	2.0906	-109.3630	-76.1150	-2.6223	-2.8218
0.306	0.7616	2540	0.5583	-3.1562	-5.2764	0.7273	2.1202	-109.6437	-76.0996	-2.6338	-2.8336
0.33	0.7676	2560	0.5585	-3.1348	-5.2566	0.7259	2.1218	-109.4454	-75.8858	-2.6445	-2.8445
0.2535	0.7736	2580	0.5599	-3.1647	-5.3073	0.7286	2.1426	-109.9522	-76.1842	-2.6577	-2.8582
0.1934	0.7796	2600	0.5572	-3.1162	-5.2379	0.7313	2.1217	-109.2587	-75.6998	-2.6468	-2.8467
0.3203	0.7856	2620	0.5566	-3.1208	-5.2449	0.7326	2.1241	-109.3288	-75.7458	-2.6348	-2.8345
0.1668	0.7916	2640	0.5579	-3.1432	-5.2688	0.7313	2.1255	-109.5672	-75.9699	-2.6396	-2.8396
0.2633	0.7976	2660	0.5594	-3.1401	-5.2593	0.7286	2.1192	-109.4729	-75.9388	-2.6318	-2.8315
0.2894	0.8036	2680	0.5565	-3.1366	-5.2691	0.7299	2.1325	-109.5705	-75.9031	-2.6321	-2.8321
0.3126	0.8096	2700	0.5566	-3.1117	-5.2476	0.7299	2.1359	-109.3550	-75.6544	-2.6348	-2.8346
0.2361	0.8156	2720	0.5558	-3.0846	-5.2199	0.7366	2.1353	-109.0784	-75.3838	-2.6386	-2.8381
0.2897	0.8216	2740	0.5552	-3.0824	-5.2114	0.7353	2.1289	-108.9931	-75.3618	-2.6278	-2.8270
0.215	0.8276	2760	0.5567	-3.0949	-5.2278	0.7340	2.1329	-109.1573	-75.4865	-2.6290	-2.8282
0.4087	0.8336	2780	0.5559	-3.0912	-5.2269	0.7326	2.1357	-109.1482	-75.4495	-2.6283	-2.8278
0.2004	0.8396	2800	0.5549	-3.0682	-5.1997	0.7326	2.1315	-108.8762	-75.2191	-2.6290	-2.8282
0.2411	0.8456	2820	0.5533	-3.0647	-5.2013	0.7313	2.1366	-108.8922	-75.1845	-2.6233	-2.8227
0.195	0.8516	2840	0.5538	-3.0700	-5.1986	0.7313	2.1286	-108.8656	-75.2373	-2.6259	-2.8256
0.3606	0.8576	2860	0.5539	-3.0873	-5.2293	0.7326	2.1421	-109.1728	-75.4102	-2.6293	-2.8292
0.2653	0.8636	2880	0.5537	-3.0914	-5.2348	0.7353	2.1433	-109.2271	-75.4520	-2.6335	-2.8338
0.2042	0.8696	2900	0.5541	-3.0897	-5.2411	0.7299	2.1514	-109.2908	-75.4350	-2.6350	-2.8351
0.1597	0.8756	2920	0.5561	-3.1010	-5.2460	0.7299	2.1450	-109.3396	-75.5478	-2.6352	-2.8353
0.2362	0.8816	2940	0.5555	-3.0924	-5.2433	0.7313	2.1509	-109.3124	-75.4613	-2.6322	-2.8325
0.2178	0.8876	2960	0.5551	-3.1061	-5.2593	0.7340	2.1532	-109.4723	-75.5984	-2.6329	-2.8331
0.19	0.8936	2980	0.5542	-3.1066	-5.2599	0.7313	2.1533	-109.4783	-75.6037	-2.6340	-2.8341
0.2363	0.8996	3000	0.5553	-3.1036	-5.2622	0.7273	2.1586	-109.5020	-75.5738	-2.6403	-2.8407
0.2951	0.9055	3020	0.5557	-3.1053	-5.2595	0.7340	2.1542	-109.4748	-75.5910	-2.6383	-2.8384
0.3123	0.9115	3040	0.5555	-3.1079	-5.2595	0.7340	2.1517	-109.4750	-75.6164	-2.6438	-2.8441
0.4072	0.9175	3060	0.5546	-3.1048	-5.2552	0.7313	2.1504	-109.4317	-75.5853	-2.6381	-2.8383
0.1172	0.9235	3080	0.5558	-3.0972	-5.2476	0.7340	2.1504	-109.3555	-75.5091	-2.6363	-2.8368
0.3237	0.9295	3100	0.5542	-3.0925	-5.2486	0.7353	2.1561	-109.3654	-75.4623	-2.6364	-2.8366
0.1755	0.9355	3120	0.5551	-3.1005	-5.2516	0.7366	2.1511	-109.3956	-75.5422	-2.6373	-2.8373
0.1999	0.9415	3140	0.5555	-3.0954	-5.2523	0.7366	2.1569	-109.4023	-75.4911	-2.6368	-2.8369
0.2447	0.9475	3160	0.5551	-3.1022	-5.2604	0.7353	2.1581	-109.4831	-75.5598	-2.6337	-2.8338
0.2232	0.9535	3180	0.5546	-3.0990	-5.2550	0.7353	2.1560	-109.4296	-75.5276	-2.6372	-2.8373
0.3249	0.9595	3200	0.5554	-3.0960	-5.2539	0.7366	2.1579	-109.4187	-75.4978	-2.6345	-2.8348
0.1206	0.9655	3220	0.5539	-3.1038	-5.2619	0.7406	2.1581	-109.4990	-75.5757	-2.6353	-2.8354
0.1695	0.9715	3240	0.5550	-3.1021	-5.2554	0.7340	2.1533	-109.4337	-75.5589	-2.6340	-2.8341
0.3007	0.9775	3260	0.5542	-3.1018	-5.2568	0.7393	2.1550	-109.4480	-75.5558	-2.6341	-2.8341
0.1249	0.9835	3280	0.5554	-3.1016	-5.2557	0.7340	2.1541	-109.4364	-75.5537	-2.6348	-2.8351
0.2911	0.9895	3300	0.5559	-3.1030	-5.2526	0.7366	2.1496	-109.4058	-75.5680	-2.6331	-2.8333
0.3547	0.9955	3320	0.5541	-3.0986	-5.2548	0.7380	2.1562	-109.4275	-75.5232	-2.6373	-2.8373

Framework versions

Transformers 4.45.2
Pytorch 2.5.1+cu121
Datasets 3.5.0
Tokenizers 0.20.3

YuchenLi01/generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.1_42

generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.1_42

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

YuchenLi01/
generatedMoreUniqueResponseNoGTv2_Qwen2.5-1.5BInstruct_dpo_ebs32_lr1e-06_beta0.1_42