You can deploy this model and start earning money today!

dmariko/
SmolLM-360M-Instruct-dpo-15k

Name: SmolLM-360M-Instruct-dpo-15k
Availability: InStock

generated_from_trainer

HuggingFaceTB/SmolLM-360M-Instruct

finetune

cc-by-nc-4.0

Views

No views yet

SmolLM-360M-Instruct-dpo-15k

This model is a fine-tuned version of HuggingFaceTB/SmolLM-360M-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4560
Rewards/chosen: 0.2819
Rewards/rejected: -0.2878
Rewards/accuracies: 0.9965
Rewards/margins: 0.5697
Logps/rejected: -448.2106
Logps/chosen: -355.1467
Logits/rejected: 0.0317
Logits/chosen: 0.4702

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 2
num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5339	0.9998	2803	0.4749	0.2598	-0.2564	0.9903	0.5162	-447.8967	-355.3675	0.0216	0.4477
0.4606	2.0	5607	0.4562	0.2846	-0.2845	0.9965	0.5692	-448.1779	-355.1194	0.0183	0.4461
0.4541	2.9998	8410	0.4552	0.2844	-0.2876	0.9956	0.5720	-448.2084	-355.1217	-0.0005	0.4160
0.4531	4.0	11214	0.4558	0.2825	-0.2877	0.9947	0.5703	-448.2096	-355.1400	-0.0151	0.3954
0.4531	4.9998	14017	0.4559	0.2816	-0.2884	0.9942	0.5700	-448.2164	-355.1490	0.0323	0.4706
0.4536	5.9989	16818	0.4560	0.2819	-0.2878	0.9965	0.5697	-448.2106	-355.1467	0.0317	0.4702

Framework versions

Transformers 4.41.0
Pytorch 2.2.0
Datasets 2.19.1
Tokenizers 0.19.1

dmariko/SmolLM-360M-Instruct-dpo-15k

SmolLM-360M-Instruct-dpo-15k

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

dmariko/
SmolLM-360M-Instruct-dpo-15k