This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the judol_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0245

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 4e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 2
total_train_batch_size: 8
total_eval_batch_size: 8
optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.15
num_epochs: 4.0
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0910	200	0.1707
No log	0.1819	400	0.1128
0.178	0.2729	600	0.0717
0.178	0.3638	800	0.1211
0.0834	0.4548	1000	0.0622
0.0834	0.5457	1200	0.0695
0.0834	0.6367	1400	0.0471
0.0628	0.7276	1600	0.0440
0.0628	0.8186	1800	0.0447
0.0567	0.9095	2000	0.0473
0.0567	1.0005	2200	0.0355
0.0567	1.0914	2400	0.0415
0.0397	1.1824	2600	0.0408
0.0397	1.2733	2800	0.0362
0.0309	1.3643	3000	0.0278
0.0309	1.4552	3200	0.0357
0.0309	1.5462	3400	0.0292
0.028	1.6371	3600	0.0296
0.028	1.7281	3800	0.0381
0.0327	1.8190	4000	0.0275
0.0327	1.9100	4200	0.0455
0.0327	2.0009	4400	0.0277
0.0277	2.0919	4600	0.0271
0.0277	2.1828	4800	0.0245
0.0241	2.2738	5000	0.0325
0.0241	2.3647	5200	0.0325
0.0241	2.4557	5400	0.0284
0.0174	2.5466	5600	0.0287
0.0174	2.6376	5800	0.0316
0.0226	2.7285	6000	0.0283
0.0226	2.8195	6200	0.0253
0.0226	2.9104	6400	0.0288
0.0221	3.0014	6600	0.0286
0.0221	3.0923	6800	0.0293
0.0166	3.1833	7000	0.0305
0.0166	3.2742	7200	0.0284
0.0166	3.3652	7400	0.0278
0.0161	3.4561	7600	0.0287
0.0161	3.5471	7800	0.0269
0.0172	3.6380	8000	0.0274
0.0172	3.7290	8200	0.0274
0.0172	3.8199	8400	0.0275
0.0147	3.9109	8600	0.0277

Framework versions

PEFT 0.18.1
Transformers 4.57.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.1

hermitcrabjr15/fine-tuned-Llama-3.2-1B-v16

fine-tuned-Llama-3.2-1B-v16

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hermitcrabjr15/
fine-tuned-Llama-3.2-1B-v16