Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
No log	0	0	1.4728	0
1.5728	0.0064	5	1.4690	24085
1.513	0.0128	10	1.4419	47453
1.3347	0.0192	15	1.3782	70909
1.3476	0.0255	20	1.3269	95452
1.4852	0.0319	25	1.2815	119981
1.1471	0.0383	30	1.2480	144786
1.3088	0.0447	35	1.2220	166464
1.201	0.0511	40	1.1959	190744
1.124	0.0575	45	1.1711	214420
1.0562	0.0638	50	1.1535	239168
1.3103	0.0702	55	1.1458	264823
1.2666	0.0766	60	1.1417	286887
1.0661	0.0830	65	1.1398	310195
1.1574	0.0894	70	1.1363	332735
1.1514	0.0958	75	1.1353	355198
1.193	0.1021	80	1.1345	376716
1.1345	0.1085	85	1.1324	400910
1.1844	0.1149	90	1.1295	424364
1.121	0.1213	95	1.1292	448878
1.1053	0.1277	100	1.1282	473220
1.3433	0.1341	105	1.1270	494797
1.0394	0.1405	110	1.1257	516472
1.0004	0.1468	115	1.1248	542496
1.063	0.1532	120	1.1224	567455
1.0935	0.1596	125	1.1211	589914
1.1756	0.1660	130	1.1200	615559
1.1847	0.1724	135	1.1195	639427
0.9203	0.1788	140	1.1191	663647
1.2464	0.1851	145	1.1177	686562
1.2849	0.1915	150	1.1166	711202
1.0506	0.1979	155	1.1145	736818
1.0547	0.2043	160	1.1137	760835
1.2118	0.2107	165	1.1136	785946
1.0628	0.2171	170	1.1134	809541
1.2405	0.2234	175	1.1132	834219
1.0144	0.2298	180	1.1124	857046
1.0397	0.2362	185	1.1114	878565
0.97	0.2426	190	1.1116	900602
1.1232	0.2490	195	1.1117	922378
1.2127	0.2554	200	1.1109	947077
1.1946	0.2618	205	1.1105	971080
1.0858	0.2681	210	1.1092	994354
1.0305	0.2745	215	1.1081	1020247
1.2059	0.2809	220	1.1073	1044169
1.1007	0.2873	225	1.1078	1068670
1.2074	0.2937	230	1.1073	1092119
1.2036	0.3001	235	1.1073	1116057
1.1312	0.3064	240	1.1060	1139741
1.356	0.3128	245	1.1065	1161776
1.2383	0.3192	250	1.1050	1186457
0.9792	0.3256	255	1.1047	1209960
1.3327	0.3320	260	1.1044	1234993
1.224	0.3384	265	1.1037	1258158
1.1602	0.3447	270	1.1030	1281170
1.1478	0.3511	275	1.1019	1303543
1.0506	0.3575	280	1.1015	1327641
1.1367	0.3639	285	1.1015	1350399
0.9812	0.3703	290	1.1008	1375625
0.9715	0.3767	295	1.1010	1401283
1.1205	0.3831	300	1.1011	1424494
0.9563	0.3894	305	1.1005	1448682
1.2123	0.3958	310	1.1006	1473827
1.2588	0.4022	315	1.0996	1496238
1.2084	0.4086	320	1.0994	1520282
1.146	0.4150	325	1.0995	1544117
1.2708	0.4214	330	1.0985	1568119
1.063	0.4277	335	1.0983	1589956
1.1589	0.4341	340	1.0984	1613080
1.101	0.4405	345	1.0993	1636206
1.101	0.4469	350	1.0985	1658755
1.3876	0.4533	355	1.0977	1681219
1.0889	0.4597	360	1.0976	1705600
1.1535	0.4660	365	1.0976	1730155
1.2215	0.4724	370	1.0976	1752902
1.1059	0.4788	375	1.0975	1776823
1.0943	0.4852	380	1.0970	1801284
1.1276	0.4916	385	1.0966	1825215
1.4414	0.4980	390	1.0968	1849301
1.0202	0.5043	395	1.0963	1873516
0.9466	0.5107	400	1.0962	1898133
1.172	0.5171	405	1.0956	1922443
1.1545	0.5235	410	1.0949	1946674
1.2317	0.5299	415	1.0949	1969275
1.3125	0.5363	420	1.0953	1991043
1.0947	0.5427	425	1.0942	2015867
1.2208	0.5490	430	1.0941	2038041
1.1131	0.5554	435	1.0945	2060342
1.0515	0.5618	440	1.0947	2084200
1.0103	0.5682	445	1.0945	2107998
1.2081	0.5746	450	1.0940	2129784
1.0893	0.5810	455	1.0926	2155055
1.0361	0.5873	460	1.0920	2179095
1.195	0.5937	465	1.0916	2201382
1.1477	0.6001	470	1.0914	2223137
1.1631	0.6065	475	1.0926	2245234
1.1492	0.6129	480	1.0912	2269870
1.1192	0.6193	485	1.0908	2294736
1.1139	0.6256	490	1.0905	2320091
1.3045	0.6320	495	1.0903	2343005
1.1927	0.6384	500	1.0902	2368849
1.0113	0.6448	505	1.0906	2392477
0.987	0.6512	510	1.0903	2415916
0.9788	0.6576	515	1.0896	2440778
1.1421	0.6640	520	1.0891	2464786
1.3182	0.6703	525	1.0887	2487622
1.1411	0.6767	530	1.0884	2511201
1.2706	0.6831	535	1.0879	2535535
1.2678	0.6895	540	1.0877	2556715
1.0137	0.6959	545	1.0874	2580524
1.1116	0.7023	550	1.0872	2605345
1.1862	0.7086	555	1.0869	2628955
1.2881	0.7150	560	1.0868	2651931
1.0775	0.7214	565	1.0856	2676797
1.2833	0.7278	570	1.0858	2700355
1.1513	0.7342	575	1.0862	2722499
1.0315	0.7406	580	1.0862	2745577
1.0138	0.7469	585	1.0852	2770551
0.9586	0.7533	590	1.0850	2794752
1.0669	0.7597	595	1.0848	2818575
1.0782	0.7661	600	1.0849	2841338
1.0795	0.7725	605	1.0852	2865516
1.2159	0.7789	610	1.0852	2887902
0.9433	0.7853	615	1.0849	2912838
1.2748	0.7916	620	1.0853	2936217
1.285	0.7980	625	1.0852	2959937
1.1652	0.8044	630	1.0855	2984114
1.0269	0.8108	635	1.0846	3007557
1.1009	0.8172	640	1.0845	3031187
1.3179	0.8236	645	1.0839	3054281
0.9786	0.8299	650	1.0832	3081188
1.4264	0.8363	655	1.0834	3104440
1.3194	0.8427	660	1.0832	3128974
1.0863	0.8491	665	1.0840	3150918
1.3073	0.8555	670	1.0844	3172136
1.0008	0.8619	675	1.0842	3194331
1.1083	0.8682	680	1.0836	3216425
1.2404	0.8746	685	1.0830	3239182
1.0338	0.8810	690	1.0824	3263452
1.1239	0.8874	695	1.0822	3288221
0.9939	0.8938	700	1.0813	3313512
0.9771	0.9002	705	1.0808	3339071
1.1427	0.9066	710	1.0806	3363039
0.9692	0.9129	715	1.0809	3386567
1.0297	0.9193	720	1.0805	3411415
1.1646	0.9257	725	1.0800	3437441
1.1262	0.9321	730	1.0802	3459420
1.0678	0.9385	735	1.0810	3483604
1.1839	0.9449	740	1.0810	3508111
1.1017	0.9512	745	1.0803	3530644
1.1201	0.9576	750	1.0797	3556161
1.1003	0.9640	755	1.0791	3578193
1.0523	0.9704	760	1.0791	3602419
1.1821	0.9768	765	1.0789	3626838
1.0863	0.9832	770	1.0790	3651332
1.3074	0.9895	775	1.0789	3677439
1.0503	0.9959	780	1.0794	3701297

Framework versions

Transformers 4.44.0
Pytorch 2.4.0+cu121
Datasets 2.20.0
Tokenizers 0.19.1

leonat3t/collapse_gemma-2-2b_hs2_massive_iter1_sftsd0

collapse_gemma-2-2b_hs2_massive_iter1_sftsd0

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

leonat3t/
collapse_gemma-2-2b_hs2_massive_iter1_sftsd0