Training Loss	Epoch	Step	Validation Loss
3.8983	0.01	25	3.8553
3.6791	0.02	50	3.6288
3.3856	0.02	75	3.2986
3.0847	0.03	100	3.0762
3.0266	0.04	125	2.9288
2.7862	0.05	150	2.8476
2.7069	0.06	175	2.7920
2.8305	0.06	200	2.7516
2.7705	0.07	225	2.7169
2.7147	0.08	250	2.6919
2.6389	0.09	275	2.6717
2.6475	0.1	300	2.6520
2.6529	0.1	325	2.6390
2.6184	0.11	350	2.6265
2.5622	0.12	375	2.6185
2.633	0.13	400	2.6069
2.599	0.14	425	2.5981
2.546	0.14	450	2.5887
2.6289	0.15	475	2.5820
2.6286	0.16	500	2.5784
2.5407	0.17	525	2.5679
2.5188	0.18	550	2.5650
2.6405	0.18	575	2.5611
2.5413	0.19	600	2.5600
2.5519	0.2	625	2.5534
2.5961	0.21	650	2.5516
2.5154	0.22	675	2.5449
2.5137	0.22	700	2.5434
2.533	0.23	725	2.5398
2.4439	0.24	750	2.5345
2.5153	0.25	775	2.5340
2.5624	0.26	800	2.5304
2.5224	0.26	825	2.5266
2.4942	0.27	850	2.5242
2.4953	0.28	875	2.5205
2.4524	0.29	900	2.5203
2.6528	0.3	925	2.5170
2.5991	0.3	950	2.5139
2.5381	0.31	975	2.5150
2.4272	0.32	1000	2.5122
2.5282	0.33	1025	2.5100
2.4874	0.34	1050	2.5098
2.4774	0.34	1075	2.5033
2.4763	0.35	1100	2.5026
2.4181	0.36	1125	2.5022
2.5109	0.37	1150	2.5002
2.4735	0.38	1175	2.4980
2.4494	0.38	1200	2.4980
2.409	0.39	1225	2.4968
2.5744	0.4	1250	2.4966
2.3832	0.41	1275	2.4960
2.4655	0.42	1300	2.4943
2.5935	0.42	1325	2.4896
2.466	0.43	1350	2.4881
2.4694	0.44	1375	2.4909
2.5194	0.45	1400	2.4889
2.5031	0.46	1425	2.4888
2.4452	0.46	1450	2.4870
2.5083	0.47	1475	2.4846
2.4407	0.48	1500	2.4840
2.4768	0.49	1525	2.4829
2.5227	0.5	1550	2.4851
2.473	0.5	1575	2.4832
2.5159	0.51	1600	2.4819
2.4545	0.52	1625	2.4798
2.3983	0.53	1650	2.4816
2.4108	0.54	1675	2.4782
2.4274	0.54	1700	2.4780
2.3832	0.55	1725	2.4767
2.5213	0.56	1750	2.4746
2.4138	0.57	1775	2.4755
2.5278	0.58	1800	2.4735
2.5962	0.58	1825	2.4754
2.4684	0.59	1850	2.4749
2.3673	0.6	1875	2.4741
2.4682	0.61	1900	2.4716
2.4858	0.62	1925	2.4690
2.4868	0.62	1950	2.4729
2.4707	0.63	1975	2.4708
2.5625	0.64	2000	2.4714
2.4543	0.65	2025	2.4728
2.4311	0.66	2050	2.4678
2.4567	0.66	2075	2.4692
2.4956	0.67	2100	2.4688
2.4889	0.68	2125	2.4698
2.5136	0.69	2150	2.4690
2.4911	0.7	2175	2.4663
2.5116	0.7	2200	2.4673
2.4444	0.71	2225	2.4681
2.4031	0.72	2250	2.4681
2.4174	0.73	2275	2.4639
2.4382	0.74	2300	2.4643
2.512	0.74	2325	2.4672
2.4861	0.75	2350	2.4641
2.4557	0.76	2375	2.4617
2.4462	0.77	2400	2.4628
2.4978	0.78	2425	2.4610
2.3829	0.78	2450	2.4614
2.5326	0.79	2475	2.4590
2.3938	0.8	2500	2.4605
2.42	0.81	2525	2.4614
2.4462	0.82	2550	2.4625
2.3891	0.82	2575	2.4626
2.5055	0.83	2600	2.4613

Framework versions

Transformers 4.40.0.dev0
Pytorch 2.1.1+cu121
Datasets 2.15.0
Tokenizers 0.15.2

thrunlab/sparse_llama_7b_hf2_refined_web_90p_2024-03-28

sparse_llama_7b_hf2_refined_web_90p_2024-03-28

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

thrunlab/
sparse_llama_7b_hf2_refined_web_90p_2024-03-28