This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the judol_train dataset. It achieves the following results on the evaluation set:

Loss: 2.5285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 6e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 2
total_train_batch_size: 8
total_eval_batch_size: 8
optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3.0
mixed_precision_training: Native AMP
label_smoothing_factor: 0.08

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0910	200	3.3010
No log	0.1819	400	2.9831
3.4599	0.2729	600	2.9014
3.4599	0.3638	800	2.7123
2.7696	0.4548	1000	2.6127
2.7696	0.5457	1200	2.5909
2.7696	0.6367	1400	2.5931
2.6119	0.7276	1600	2.5797
2.6119	0.8186	1800	2.5534
2.5769	0.9095	2000	2.5617
2.5769	1.0005	2200	2.5463
2.5769	1.0914	2400	2.5504
2.557	1.1824	2600	2.5458
2.557	1.2733	2800	2.5623
2.543	1.3643	3000	2.5319
2.543	1.4552	3200	2.5520
2.543	1.5462	3400	2.5412
2.5295	1.6371	3600	2.5422
2.5295	1.7281	3800	2.5428
2.5338	1.8190	4000	2.5330
2.5338	1.9100	4200	2.5617
2.5338	2.0009	4400	2.5413
2.5297	2.0919	4600	2.5414
2.5297	2.1828	4800	2.5285
2.5318	2.2738	5000	2.5492
2.5318	2.3647	5200	2.5381
2.5318	2.4557	5400	2.5360
2.5131	2.5466	5600	2.5394
2.5131	2.6376	5800	2.5359
2.5232	2.7285	6000	2.5341
2.5232	2.8195	6200	2.5344
2.5232	2.9104	6400	2.5348

Framework versions

PEFT 0.18.1
Transformers 4.57.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.1

hermitcrabjr15/fine-tuned-Llama-3.2-1B-v7

fine-tuned-Llama-3.2-1B-v7

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hermitcrabjr15/
fine-tuned-Llama-3.2-1B-v7