Training Loss	Epoch	Step	Validation Loss
2.5973	0.0	25	2.4361
2.3245	0.01	50	2.4253
2.3668	0.01	75	2.4063
2.3521	0.02	100	2.3820
2.3119	0.02	125	2.3613
2.4636	0.02	150	2.3452
2.3372	0.03	175	2.3331
2.2927	0.03	200	2.3267
2.3831	0.04	225	2.3196
2.4013	0.04	250	2.3158
2.2576	0.04	275	2.3109
2.2803	0.05	300	2.3104
2.3531	0.05	325	2.3070
2.0961	0.06	350	2.3066
2.2809	0.06	375	2.3042
2.4516	0.06	400	2.3034
2.3927	0.07	425	2.3027
2.3188	0.07	450	2.3030
2.2165	0.08	475	2.3028
2.2814	0.08	500	2.3003
2.3224	0.08	525	2.3005
2.2267	0.09	550	2.2991
2.3238	0.09	575	2.2976
2.1845	0.1	600	2.2992
2.2896	0.1	625	2.2994
2.334	0.1	650	2.2973
2.3298	0.11	675	2.2972
2.2847	0.11	700	2.2950
2.2589	0.12	725	2.2966
2.1715	0.12	750	2.2949
2.3554	0.12	775	2.2940
2.3217	0.13	800	2.2970
2.3374	0.13	825	2.2940
2.2951	0.14	850	2.2933
2.1613	0.14	875	2.2934
2.278	0.14	900	2.2928
2.1964	0.15	925	2.2930
2.3489	0.15	950	2.2926
2.3561	0.16	975	2.2919
2.3854	0.16	1000	2.2907
2.3849	0.16	1025	2.2931
2.2953	0.17	1050	2.2917
2.3519	0.17	1075	2.2901
2.2702	0.18	1100	2.2927
2.2834	0.18	1125	2.2888
2.4015	0.18	1150	2.2925
2.3967	0.19	1175	2.2915
2.1978	0.19	1200	2.2892
2.2758	0.2	1225	2.2886
2.2774	0.2	1250	2.2905
2.3803	0.2	1275	2.2930
2.3233	0.21	1300	2.2909
2.2431	0.21	1325	2.2908
2.2042	0.22	1350	2.2885
2.3585	0.22	1375	2.2885
2.2779	0.22	1400	2.2906
2.2119	0.23	1425	2.2883
2.2257	0.23	1450	2.2876
2.2383	0.24	1475	2.2892
2.2166	0.24	1500	2.2899
2.1817	0.24	1525	2.2872
2.3113	0.25	1550	2.2880
2.3212	0.25	1575	2.2876
2.3027	0.26	1600	2.2838
2.2737	0.26	1625	2.2888
2.3181	0.26	1650	2.2865
2.2759	0.27	1675	2.2871
2.3179	0.27	1700	2.2858
2.3111	0.28	1725	2.2862
2.2699	0.28	1750	2.2872
2.3006	0.28	1775	2.2859
2.1913	0.29	1800	2.2849
2.3133	0.29	1825	2.2852
2.286	0.3	1850	2.2849
2.3403	0.3	1875	2.2860
2.2713	0.3	1900	2.2843
2.3492	0.31	1925	2.2840
2.2747	0.31	1950	2.2840
2.2783	0.32	1975	2.2841
2.1854	0.32	2000	2.2836
2.2544	0.32	2025	2.2826
2.2955	0.33	2050	2.2837
2.2053	0.33	2075	2.2849
2.2848	0.34	2100	2.2841
2.4168	0.34	2125	2.2843
2.268	0.34	2150	2.2814
2.2045	0.35	2175	2.2841
2.2717	0.35	2200	2.2825
2.1958	0.36	2225	2.2827
2.1635	0.36	2250	2.2867
2.3043	0.36	2275	2.2820
2.3106	0.37	2300	2.2845
2.1992	0.37	2325	2.2829
2.2189	0.38	2350	2.2817
2.3096	0.38	2375	2.2830
2.2803	0.38	2400	2.2839
2.1752	0.39	2425	2.2825
2.1324	0.39	2450	2.2834
2.207	0.4	2475	2.2846
2.3369	0.4	2500	2.2844
2.3512	0.4	2525	2.2853
2.1432	0.41	2550	2.2861
2.1743	0.41	2575	2.2832
2.2696	0.42	2600	2.2832

Framework versions

Transformers 4.40.0.dev0
Pytorch 2.1.1+cu121
Datasets 2.15.0
Tokenizers 0.15.2

thrunlab/sparse_llama_7b_hf2_refined_web_70p_2024-03-28

sparse_llama_7b_hf2_refined_web_70p_2024-03-28

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

thrunlab/
sparse_llama_7b_hf2_refined_web_70p_2024-03-28