You can deploy this model and start earning money today!

cj453/
NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

Name: NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh
Availability: InStock

safetensors

opt

generated_from_trainer

facebook/opt-1.3b

finetune

other

Views

No views yet

NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

This model is a fine-tuned version of facebook/opt-1.3b on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.2160
Accuracy: 0.6289
Train Rewards/chosen: 13.3266
Train Rewards/rejected: -10.7412
Train Rewards/accuracies: 0.9925
Train Rewards/margins: 24.0678
Train Nll Loss: 1.9271
Train Logit Total Loss: 0.0395
Train Logit Loss: 0.0204
Rewards/chosen: 4.7138
Rewards/rejected: -1.7686
Rewards/accuracies: 0.6145
Rewards/margins: 6.4823
Nll Loss: 2.0087
Logit Total Loss: 3.2131
Logit Loss: 3.2252

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1.41e-05
train_batch_size: 4
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Nll Loss	Logit Total Loss	Logit Loss
0.7879	0.11	100	0.7507	0.4845	-0.1740	-0.1876	0.4714	0.0135	6.2659	0.7498	0.6941
0.7291	0.23	200	0.7331	0.5773	-0.2697	-0.3880	0.5645	0.1184	6.0096	0.7310	0.6777
0.6843	0.34	300	0.7057	0.5876	0.2058	-0.0389	0.5754	0.2448	3.9577	0.7039	0.6710
0.6773	0.46	400	0.6950	0.5918	-0.0097	-0.2138	0.5774	0.2041	4.2789	0.6968	0.6607
0.7071	0.57	500	0.7107	0.5918	0.7447	0.5198	0.5815	0.2249	4.4170	0.7087	0.6712
0.6881	0.69	600	0.6687	0.6186	-0.8010	-1.0541	0.6028	0.2531	3.2753	0.6671	0.6408
0.6871	0.8	700	0.6847	0.5753	-1.7064	-1.9330	0.5601	0.2266	3.7264	0.6839	0.6532
0.7125	0.91	800	0.6885	0.5814	-1.4574	-1.6521	0.5734	0.1947	4.3386	0.6851	0.6482
0.62	1.03	900	0.6955	0.6103	-1.4133	-1.8571	0.5964	0.4438	3.1332	0.6958	0.6712
0.5929	1.14	1000	0.6537	0.6371	-1.9413	-2.5254	0.6226	0.5841	2.9107	0.6524	0.6296
0.5825	1.26	1100	0.6749	0.6515	0.4669	-0.0787	0.6367	0.5455	2.9364	0.6732	0.6503
0.614	1.37	1200	0.6697	0.6351	0.1785	-0.2933	0.6258	0.4718	2.9997	0.6659	0.6423
0.5528	1.49	1300	0.6553	0.6268	-1.0780	-1.6306	0.6177	0.5526	2.9051	0.6504	0.6276
0.6501	1.6	1400	0.6379	0.6412	-1.6259	-2.1203	0.6306	0.4944	2.9085	0.6351	0.6121
0.545	1.71	1500	0.6640	0.6660	-0.3375	-1.1855	0.6560	0.8480	3.0934	0.6573	0.6327
0.6653	1.83	1600	0.6379	0.6639	-1.0663	-1.6961	0.6528	0.6298	2.8475	0.6376	0.6153
0.5792	1.94	1700	0.6447	0.6577	-0.0283	-0.6093	0.6480	0.5810	3.0457	0.6420	0.6178
0.2858	2.06	1800	0.9327	0.6330	1.7576	0.2131	0.6226	1.5445	2.8731	0.9216	0.9019
0.3404	2.17	1900	0.8438	0.6144	0.9326	-0.2443	0.6024	1.1769	2.7925	0.8418	0.8221
0.2734	2.29	2000	0.9082	0.6227	1.6779	0.4268	0.6125	1.2511	2.7991	0.8986	0.8794
0.2562	2.4	2100	0.9566	0.6371	2.2122	0.5184	0.6266	1.6937	2.7729	0.9522	0.9338
0.3796	2.51	2200	0.8839	0.6351	0.7900	-0.5311	0.6218	1.3211	2.7689	0.8720	0.8528
0.2316	2.63	2300	0.8741	0.6454	2.0133	0.5784	0.6359	1.4349	2.7465	0.8633	0.8443
0.3679	2.74	2400	0.8584	0.6515	-0.8628	-2.1801	0.6379	1.3173	2.7134	0.8483	0.8294
0.3384	2.86	2500	0.9165	0.6412	-0.9835	-2.3685	0.6294	1.3850	2.7084	0.9087	0.8905
0.3595	2.97	2600	0.9173	0.6454	0.3307	-1.0129	0.6347	1.3436	2.7089	0.9049	0.8867
0.1331	3.09	2700	1.4595	0.6557	0.6119	-2.1780	0.6468	2.7900	2.6967	1.4381	1.4254
0.1464	3.2	2800	1.4234	0.6351	5.4974	2.9945	0.6258	2.5029	2.6392	1.3999	1.3874
0.137	3.31	2900	1.4612	0.6474	3.1356	0.4400	0.6363	2.6956	2.6002	1.4435	1.4318
0.1593	3.43	3000	1.7826	0.6433	3.8280	0.7687	0.6282	3.0593	2.6206	1.7676	1.7590
0.0834	3.54	3100	1.5493	0.6474	2.4447	-0.2971	0.6355	2.7418	2.6296	1.5386	1.5275
0.136	3.66	3200	1.5847	0.6495	2.6691	-0.1416	0.6375	2.8108	2.6007	1.5701	1.5597
0.0859	3.77	3300	1.7114	0.6227	0.8690	-1.9033	0.6093	2.7723	2.5630	1.6942	1.6854
0.1705	3.89	3400	1.7792	0.6268	-1.4030	-4.0698	0.6121	2.6669	2.5917	1.7786	1.7704
0.1675	4.0	3500	2.1762	0.6268	-1.5886	-5.0180	0.6133	3.4294	2.5716	2.1579	2.1537
0.0589	4.11	3600	2.3409	0.6309	1.1330	-2.8993	0.6173	4.0323	2.4949	2.3055	2.3036
0.1014	4.23	3700	2.3221	0.6268	2.6255	-1.3486	0.6125	3.9741	2.4617	2.2985	2.2969
0.0697	4.34	3800	2.4256	0.6351	2.8885	-1.2680	0.6194	4.1565	2.4613	2.4010	2.4004
0.1687	4.46	3900	2.1905	0.6433	3.3404	-1.0572	0.6347	4.3976	2.4074	2.1582	2.1556
0.0315	4.57	4000	2.3170	0.6619	2.0050	-2.4036	0.6480	4.4086	2.4112	2.2812	2.2799
0.1071	4.69	4100	2.2205	0.6454	0.9399	-3.4755	0.6379	4.4154	2.3561	2.1998	2.1983
0.1342	4.8	4200	2.2640	0.6557	10.1640	5.7216	0.6419	4.4424	2.3536	2.2410	2.2399
0.0793	4.91	4300	2.0629	0.6495	-0.6830	-4.8288	0.6327	4.1458	2.3658	2.0407	2.0374
0.0587	5.03	4400	2.3862	0.6371	3.2076	-1.4161	0.6258	4.6238	2.3529	2.3625	2.3626
0.0433	5.14	4500	2.5409	0.6454	4.9940	0.1253	0.6286	4.8687	2.3166	2.5250	2.5271
0.0506	5.26	4600	2.5949	0.6557	6.7660	1.6624	0.6395	5.1035	2.2864	2.5983	2.6014
0.0506	5.37	4700	2.7389	0.6351	7.2608	2.0917	0.6226	5.1690	2.2691	2.7197	2.7243
0.0644	5.49	4800	2.8523	0.6309	2.3756	-2.9285	0.6173	5.3041	2.2594	2.8574	2.8634
0.0714	5.6	4900	2.5013	0.6309	2.5445	-2.3571	0.6206	4.9016	2.2422	2.5045	2.5072
0.1087	5.71	5000	2.6378	0.6227	-0.0320	-5.0243	0.6113	4.9923	2.2318	2.6303	2.6344
0.0874	5.83	5100	2.8088	0.6412	5.9816	0.6049	0.6278	5.3767	2.2257	2.7811	2.7867
0.0871	5.94	5200	2.4819	0.6433	7.2347	2.1895	0.6306	5.0451	2.2034	2.4679	2.4706
0.0331	6.06	5300	2.8775	0.6268	9.8380	4.4195	0.6145	5.4184	2.1978	2.8663	2.8730
0.024	6.17	5400	2.8923	0.6433	5.1441	-0.5990	0.6306	5.7431	2.1912	2.8713	2.8781
0.0354	6.29	5500	2.7626	0.6433	-1.4206	-6.9376	0.6315	5.5170	2.1826	2.7519	2.7577
0.0289	6.4	5600	2.8423	0.6371	7.1683	1.7904	0.6246	5.3779	2.1707	2.8182	2.8248
0.0389	6.51	5700	2.9096	0.6412	2.0666	-3.5386	0.6234	5.6052	2.1672	2.9140	2.9215
0.0245	6.63	5800	2.8677	0.6495	4.5194	-1.1798	0.6347	5.6992	2.1466	2.8461	2.8532
0.0804	6.74	5900	2.9668	0.6371	5.6766	-0.3308	0.6226	6.0074	2.1468	2.9437	2.9518
0.029	6.86	6000	3.0269	0.6371	3.9285	-2.2229	0.6226	6.1514	2.1305	2.9998	3.0086
0.0438	6.97	6100	2.8192	0.6639	2.2607	-4.3102	0.6476	6.5708	2.1277	2.8101	2.8170
0.0451	7.09	6200	2.8547	0.6577	2.5219	-3.4933	0.6395	6.0152	2.1111	2.8383	2.8456
0.0761	7.2	6300	2.9610	0.6536	4.7705	-1.5571	0.6435	6.3275	2.1023	2.9370	2.9454
0.0477	7.31	6400	2.8708	0.6619	2.7809	-3.7082	0.6488	6.4891	2.0958	2.8410	2.8485
0.0449	7.43	6500	3.0901	0.6619	5.8808	-0.8822	0.6496	6.7630	2.0873	3.0685	3.0784
0.0418	7.54	6600	2.9687	0.6371	2.2079	-4.1264	0.6206	6.3343	2.0853	2.9514	2.9602
0.0473	7.66	6700	2.9895	0.6351	2.4455	-3.8039	0.6206	6.2494	2.0790	2.9705	2.9795
0.0459	7.77	6800	3.0660	0.6392	4.6892	-1.6980	0.6254	6.3872	2.0757	3.0540	3.0638
0.045	7.89	6900	3.0811	0.6474	2.9687	-3.4595	0.6347	6.4282	2.0697	3.0561	3.0661
0.0493	8.0	7000	2.9549	0.6330	3.3733	-2.8947	0.6214	6.2680	2.0679	2.9435	2.9523
0.031	8.11	7100	2.9964	0.6330	4.2065	-2.1412	0.6206	6.3477	2.0650	2.9810	2.9903
0.0196	8.23	7200	3.0962	0.6371	4.8289	-1.6916	0.6246	6.5204	2.0550	3.0800	3.0904
0.0223	8.34	7300	3.0038	0.6392	2.7990	-3.5327	0.6246	6.3317	2.0497	2.9870	2.9965
0.0629	8.46	7400	3.0349	0.6351	5.2916	-0.8920	0.6206	6.1836	2.0453	3.0076	3.0173
0.0922	8.57	7500	3.0735	0.6227	1.5229	-4.6388	0.6105	6.1617	2.0409	3.0489	3.0591
0.0302	8.69	7600	3.1279	0.6289	1.4324	-4.7615	0.6185	6.1939	2.0355	3.1060	3.1168
0.0589	8.8	7700	3.1274	0.6412	4.6809	-1.6469	0.6306	6.3279	2.0298	3.1051	3.1159
0.0389	8.91	7800	3.0308	0.6330	4.8002	-1.3492	0.6206	6.1494	2.0277	3.0129	3.0229
0.0252	9.03	7900	3.0680	0.6330	5.0212	-1.1246	0.6165	6.1458	2.0236	3.0565	3.0670
0.0652	9.14	8000	3.1190	0.6351	4.3150	-1.9926	0.6165	6.3076	2.0196	3.1234	3.1345
0.0201	9.26	8100	3.1413	0.6289	4.7573	-1.5726	0.6165	6.3299	2.0164	3.1389	3.1503
0.0443	9.37	8200	3.1135	0.6247	4.3945	-1.9119	0.6125	6.3065	2.0140	3.1029	3.1139
0.0186	9.49	8300	3.1597	0.6289	3.7131	-2.6943	0.6165	6.4074	2.0114	3.1487	3.1602
0.0352	9.6	8400	3.1513	0.6247	3.9594	-2.4902	0.6085	6.4496	2.0100	3.1409	3.1523
0.0225	9.71	8500	3.1966	0.6227	4.9750	-1.5016	0.6125	6.4766	2.0095	3.1854	3.1973
0.0385	9.83	8600	3.2165	0.6268	4.9076	-1.6079	0.6125	6.5155	2.0094	3.2082	3.2203
0.0266	9.94	8700	3.2160	0.6289	4.7138	-1.7686	0.6145	6.4823	2.0087	3.2131	3.2252

Framework versions

Transformers 4.37.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.15.2

cj453/NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

cj453/
NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh