This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the judol_train dataset. It achieves the following results on the evaluation set:

Loss: 1.6462

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 6e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 2
gradient_accumulation_steps: 2
total_train_batch_size: 16
total_eval_batch_size: 8
optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3.0
mixed_precision_training: Native AMP
label_smoothing_factor: 0.05

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.1819	200	1.9373
No log	0.3638	400	1.7804
4.0708	0.5457	600	1.6978
4.0708	0.7276	800	1.6812
3.3558	0.9095	1000	1.6647
3.3558	1.0910	1200	1.6597
3.3558	1.2729	1400	1.6498
3.3009	1.4548	1600	1.6523
3.3009	1.6367	1800	1.6485
3.2751	1.8186	2000	1.6701
3.2751	2.0	2200	1.6478
3.2751	2.1819	2400	1.6462
3.2719	2.3638	2600	1.6565
3.2719	2.5457	2800	1.6486
3.2494	2.7276	3000	1.6498
3.2494	2.9095	3200	1.6494

Framework versions

PEFT 0.18.1
Transformers 4.57.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.1

hermitcrabjr15/fine-tuned-Llama-3.2-1B-v13

fine-tuned-Llama-3.2-1B-v13

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hermitcrabjr15/
fine-tuned-Llama-3.2-1B-v13