This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on the t1_100k_v3_tag5_cleaned dataset. It achieves the following results on the evaluation set:

Loss: 0.2440

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
total_train_batch_size: 4
total_eval_batch_size: 4
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.3623	0.0105	100	0.3659
0.3493	0.0210	200	0.3450
0.3537	0.0315	300	0.3406
0.3635	0.0421	400	0.3291
0.4105	0.0526	500	0.3243
0.3187	0.0631	600	0.3198
0.2956	0.0736	700	0.3184
0.3243	0.0841	800	0.3098
0.3029	0.0946	900	0.3072
0.3726	0.1052	1000	0.3024
0.2796	0.1157	1100	0.3052
0.3258	0.1262	1200	0.2978
0.3175	0.1367	1300	0.2993
0.3014	0.1472	1400	0.2988
0.323	0.1577	1500	0.2946
0.3368	0.1683	1600	0.2946
0.3736	0.1788	1700	0.2918
0.2768	0.1893	1800	0.2894
0.3027	0.1998	1900	0.2895
0.2689	0.2103	2000	0.2881
0.3747	0.2208	2100	0.2883
0.2667	0.2314	2200	0.2869
0.3388	0.2419	2300	0.2819
0.2394	0.2524	2400	0.2854
0.2967	0.2629	2500	0.2835
0.2581	0.2734	2600	0.2805
0.2497	0.2839	2700	0.2771
0.2833	0.2945	2800	0.2786
0.2979	0.3050	2900	0.2783
0.3042	0.3155	3000	0.2776
0.2949	0.3260	3100	0.2770
0.27	0.3365	3200	0.2764
0.2823	0.3470	3300	0.2747
0.281	0.3576	3400	0.2732
0.3614	0.3681	3500	0.2709
0.2892	0.3786	3600	0.2725
0.3151	0.3891	3700	0.2710
0.3496	0.3996	3800	0.2700
0.3053	0.4101	3900	0.2700
0.2913	0.4207	4000	0.2701
0.2639	0.4312	4100	0.2687
0.2613	0.4417	4200	0.2678
0.2769	0.4522	4300	0.2651
0.2616	0.4627	4400	0.2653
0.3045	0.4732	4500	0.2650
0.2994	0.4838	4600	0.2625
0.2792	0.4943	4700	0.2618
0.2558	0.5048	4800	0.2621
0.2325	0.5153	4900	0.2605
0.2679	0.5258	5000	0.2600
0.2553	0.5363	5100	0.2586
0.2738	0.5469	5200	0.2588
0.2712	0.5574	5300	0.2574
0.278	0.5679	5400	0.2574
0.2775	0.5784	5500	0.2573
0.2898	0.5889	5600	0.2571
0.2219	0.5994	5700	0.2553
0.2654	0.6099	5800	0.2551
0.2653	0.6205	5900	0.2546
0.2481	0.6310	6000	0.2550
0.2581	0.6415	6100	0.2540
0.2263	0.6520	6200	0.2542
0.2562	0.6625	6300	0.2530
0.2448	0.6730	6400	0.2524
0.228	0.6836	6500	0.2520
0.2932	0.6941	6600	0.2508
0.2261	0.7046	6700	0.2499
0.2589	0.7151	6800	0.2507
0.2237	0.7256	6900	0.2496
0.2725	0.7361	7000	0.2484
0.2641	0.7467	7100	0.2492
0.2795	0.7572	7200	0.2481
0.2097	0.7677	7300	0.2484
0.2027	0.7782	7400	0.2476
0.277	0.7887	7500	0.2473
0.2732	0.7992	7600	0.2470
0.2603	0.8098	7700	0.2462
0.2428	0.8203	7800	0.2466
0.2052	0.8308	7900	0.2458
0.2408	0.8413	8000	0.2457
0.2401	0.8518	8100	0.2456
0.2661	0.8623	8200	0.2451
0.2376	0.8729	8300	0.2449
0.2381	0.8834	8400	0.2446
0.2389	0.8939	8500	0.2445
0.2385	0.9044	8600	0.2445
0.2139	0.9149	8700	0.2441
0.2977	0.9254	8800	0.2441
0.2412	0.9360	8900	0.2441
0.2583	0.9465	9000	0.2441
0.2454	0.9570	9100	0.2440
0.2232	0.9675	9200	0.2440
0.2689	0.9780	9300	0.2439
0.2405	0.9885	9400	0.2440
0.2587	0.9991	9500	0.2439

Framework versions

Transformers 4.46.1
Pytorch 2.6.0+cu124
Datasets 3.1.0
Tokenizers 0.20.3

lemonhat/Llama-3.1-8B-Instruct-t1_100k_v3_tag5_cleaned

t1_100k_v3_tag5_cleaned

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

lemonhat/
Llama-3.1-8B-Instruct-t1_100k_v3_tag5_cleaned