Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
No log	0	0	1.4728	0
1.5413	0.0064	5	1.4690	24099
1.468	0.0128	10	1.4415	47953
1.413	0.0192	15	1.3760	71852
1.8762	0.0255	20	1.3225	94861
1.3668	0.0319	25	1.2746	118551
1.415	0.0383	30	1.2466	143220
1.1385	0.0447	35	1.2198	168406
1.3159	0.0511	40	1.1959	192540
1.4691	0.0575	45	1.1762	217563
1.194	0.0638	50	1.1568	240650
1.4753	0.0702	55	1.1458	262924
0.9981	0.0766	60	1.1420	288919
1.1523	0.0830	65	1.1410	311700
1.1387	0.0894	70	1.1378	337665
1.165	0.0958	75	1.1360	360779
1.0939	0.1021	80	1.1341	384518
1.1318	0.1085	85	1.1331	407998
1.1093	0.1149	90	1.1307	430872
1.3333	0.1213	95	1.1292	456904
1.2117	0.1277	100	1.1279	480414
1.076	0.1341	105	1.1270	503612
1.2112	0.1405	110	1.1257	527097
1.2934	0.1468	115	1.1253	551465
1.0591	0.1532	120	1.1248	576515
1.2681	0.1596	125	1.1225	598661
1.3159	0.1660	130	1.1211	622520
1.0536	0.1724	135	1.1197	647066
1.4847	0.1788	140	1.1195	668693
1.2733	0.1851	145	1.1195	693930
1.0681	0.1915	150	1.1177	717354
1.0521	0.1979	155	1.1166	741477
1.1823	0.2043	160	1.1160	766719
1.25	0.2107	165	1.1158	790498
1.2179	0.2171	170	1.1157	812664
1.1066	0.2234	175	1.1162	835233
1.3625	0.2298	180	1.1141	859917
1.1624	0.2362	185	1.1130	882246
1.083	0.2426	190	1.1126	906972
1.0507	0.2490	195	1.1121	931095
1.1392	0.2554	200	1.1115	953263
1.1702	0.2618	205	1.1111	975018
1.1659	0.2681	210	1.1119	998934
1.3211	0.2745	215	1.1116	1022277
1.4559	0.2809	220	1.1114	1044095
1.1317	0.2873	225	1.1100	1067334
1.3368	0.2937	230	1.1090	1091404
1.1618	0.3001	235	1.1078	1114792
1.4909	0.3064	240	1.1072	1138163
1.1799	0.3128	245	1.1064	1160499
1.1344	0.3192	250	1.1060	1184031
1.1666	0.3256	255	1.1056	1208206
1.2509	0.3320	260	1.1058	1230718
1.1293	0.3384	265	1.1059	1253387
1.2628	0.3447	270	1.1047	1278359
1.3148	0.3511	275	1.1034	1301171
1.1599	0.3575	280	1.1031	1324816
1.0796	0.3639	285	1.1029	1347879
1.5384	0.3703	290	1.1026	1371212
1.0318	0.3767	295	1.1028	1394900
1.2716	0.3831	300	1.1015	1419596
1.006	0.3894	305	1.1011	1445055
1.2733	0.3958	310	1.1011	1469741
1.1708	0.4022	315	1.1017	1493344
1.1193	0.4086	320	1.1005	1519541
1.1854	0.4150	325	1.1005	1542273
1.1462	0.4214	330	1.0999	1565438
1.1932	0.4277	335	1.0999	1589145
1.0484	0.4341	340	1.0990	1612441
1.3366	0.4405	345	1.0988	1637433
1.2091	0.4469	350	1.0983	1660076
1.1022	0.4533	355	1.0981	1683065
0.9737	0.4597	360	1.0976	1707553
1.1405	0.4660	365	1.0975	1731523
1.251	0.4724	370	1.0974	1755850
1.0895	0.4788	375	1.0970	1780595
1.0132	0.4852	380	1.0967	1805354
1.0945	0.4916	385	1.0967	1829530
1.1195	0.4980	390	1.0958	1852450
1.0187	0.5043	395	1.0956	1876975
1.0871	0.5107	400	1.0954	1899748
1.1466	0.5171	405	1.0955	1922569
1.0075	0.5235	410	1.0953	1947856
0.9091	0.5299	415	1.0950	1971470
0.8935	0.5363	420	1.0950	1996263
1.1449	0.5427	425	1.0942	2021420
1.1541	0.5490	430	1.0940	2046199
1.151	0.5554	435	1.0940	2069146
1.3337	0.5618	440	1.0938	2091621
1.0196	0.5682	445	1.0935	2117568
0.89	0.5746	450	1.0930	2140815
1.0622	0.5810	455	1.0923	2165288
1.053	0.5873	460	1.0917	2188500
1.1988	0.5937	465	1.0911	2212697
1.3885	0.6001	470	1.0913	2236351
1.2197	0.6065	475	1.0916	2257803
1.1652	0.6129	480	1.0918	2280778
1.0595	0.6193	485	1.0916	2303535
1.062	0.6256	490	1.0905	2325459
1.1904	0.6320	495	1.0902	2349646
1.0868	0.6384	500	1.0891	2373919
1.1437	0.6448	505	1.0896	2395810
1.2257	0.6512	510	1.0898	2419873
1.1189	0.6576	515	1.0900	2444153
0.9017	0.6640	520	1.0891	2469409
1.1747	0.6703	525	1.0889	2496209
0.9622	0.6767	530	1.0882	2517507
1.3503	0.6831	535	1.0885	2541822
1.2032	0.6895	540	1.0881	2564845
0.9953	0.6959	545	1.0875	2590084
0.993	0.7023	550	1.0876	2614115
1.1117	0.7086	555	1.0876	2638856
0.9781	0.7150	560	1.0872	2664678
1.1636	0.7214	565	1.0869	2688524
1.0864	0.7278	570	1.0869	2710716
1.0888	0.7342	575	1.0869	2734994
1.0591	0.7406	580	1.0869	2757358
1.1579	0.7469	585	1.0862	2780811
1.3256	0.7533	590	1.0858	2804393
1.1933	0.7597	595	1.0862	2827623
1.0728	0.7661	600	1.0858	2853109
1.1327	0.7725	605	1.0857	2876700
1.1123	0.7789	610	1.0852	2899979
1.0265	0.7853	615	1.0847	2923934
1.2297	0.7916	620	1.0849	2946748
1.0586	0.7980	625	1.0852	2968864
1.2438	0.8044	630	1.0853	2991729
1.0378	0.8108	635	1.0849	3014593
1.1626	0.8172	640	1.0846	3038620
0.9424	0.8236	645	1.0847	3061990
1.1932	0.8299	650	1.0839	3085630
0.9673	0.8363	655	1.0835	3109369
1.0592	0.8427	660	1.0835	3132365
1.103	0.8491	665	1.0830	3155131
1.1921	0.8555	670	1.0822	3179262
1.1857	0.8619	675	1.0826	3203462
1.1784	0.8682	680	1.0831	3227442
1.0446	0.8746	685	1.0829	3252861
1.1003	0.8810	690	1.0830	3275550
1.2749	0.8874	695	1.0826	3299309
1.1357	0.8938	700	1.0821	3324346
1.064	0.9002	705	1.0818	3347276
1.0834	0.9066	710	1.0821	3372699
1.178	0.9129	715	1.0823	3396981
1.0715	0.9193	720	1.0820	3418185
1.302	0.9257	725	1.0819	3440184
1.247	0.9321	730	1.0817	3463307
1.0523	0.9385	735	1.0815	3487355
1.0212	0.9449	740	1.0814	3511446
1.144	0.9512	745	1.0816	3534506
1.2586	0.9576	750	1.0812	3558296
1.1784	0.9640	755	1.0808	3582289
1.2164	0.9704	760	1.0806	3606452
1.2791	0.9768	765	1.0807	3630506
0.9183	0.9832	770	1.0805	3655694
1.0453	0.9895	775	1.0802	3679242
1.1328	0.9959	780	1.0812	3701328

Framework versions

Transformers 4.44.0
Pytorch 2.4.0+cu121
Datasets 2.20.0
Tokenizers 0.19.1

leonat3t/collapse_gemma-2-2b_hs2_massive_iter1_sftsd2

collapse_gemma-2-2b_hs2_massive_iter1_sftsd2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

leonat3t/
collapse_gemma-2-2b_hs2_massive_iter1_sftsd2