You can deploy this model and start earning money today!

dmariko/
SmolLM-1.7B-Instruct-dpo-15k

Name: SmolLM-1.7B-Instruct-dpo-15k
Rating: 2.5 (9 reviews)

generated_from_trainer

HuggingFaceTB/SmolLM-360M-Instruct

finetune

apache-2.0

Views

No views yet

SmolLM-1.7B-Instruct-dpo-15k

This model is a fine-tuned version of HuggingFaceTB/SmolLM-360M-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4559
Rewards/chosen: 0.2769
Rewards/rejected: -0.2932
Rewards/accuracies: 0.9969
Rewards/margins: 0.5701
Logps/rejected: -448.2645
Logps/chosen: -355.1967
Logits/rejected: 0.0365
Logits/chosen: 0.4782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 2
num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5349	0.9998	2803	0.4751	0.2555	-0.2601	0.9965	0.5156	-447.9330	-355.4099	-0.0010	0.4094
0.4605	2.0	5607	0.4568	0.2750	-0.2927	0.9969	0.5677	-448.2599	-355.2158	0.0076	0.4353
0.4541	2.9998	8410	0.4548	0.2831	-0.2903	0.9947	0.5734	-448.2353	-355.1347	-0.0002	0.4193
0.4525	4.0	11214	0.4547	0.2846	-0.2888	0.9973	0.5733	-448.2202	-355.1198	-0.0289	0.3672
0.4529	4.9998	14017	0.4547	0.2811	-0.2927	0.9956	0.5738	-448.2591	-355.1540	0.0410	0.4823
0.4536	5.9989	16818	0.4559	0.2769	-0.2932	0.9969	0.5701	-448.2645	-355.1967	0.0365	0.4782

Framework versions

Transformers 4.41.0
Pytorch 2.2.0
Datasets 2.19.1
Tokenizers 0.19.1

dmariko/SmolLM-1.7B-Instruct-dpo-15k

SmolLM-1.7B-Instruct-dpo-15k

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

dmariko/
SmolLM-1.7B-Instruct-dpo-15k