You can deploy this model and start earning money today!

dmariko/
SmolLM-360M-Instruct-dpo-16k

Name: SmolLM-360M-Instruct-dpo-16k
Availability: InStock

generated_from_trainer

HuggingFaceTB/SmolLM-360M-Instruct

finetune

cc-by-nc-4.0

Views

No views yet

SmolLM-360M-Instruct-dpo-16k

This model is a fine-tuned version of HuggingFaceTB/SmolLM-360M-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.8873
Rewards/chosen: 0.0047
Rewards/rejected: 0.3539
Rewards/accuracies: 0.0326
Rewards/margins: -0.3493
Logps/rejected: -470.7575
Logps/chosen: -546.0133
Logits/rejected: 0.3014
Logits/chosen: 0.6045

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 2
num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5225	0.9999	3368	0.8679	0.0092	0.3258	0.0337	-0.3166	-471.0385	-545.9679	0.3212	0.6250
0.4511	2.0	6737	0.8863	0.0171	0.3649	0.0283	-0.3477	-470.6477	-545.8885	0.2889	0.5939
0.4453	2.9999	10105	0.8880	0.0006	0.3516	0.0304	-0.3510	-470.7807	-546.0537	0.3259	0.6291
0.4439	4.0	13474	0.8894	0.0067	0.3598	0.0228	-0.3531	-470.6990	-545.9932	0.2699	0.5815
0.4441	4.9999	16842	0.8881	0.0058	0.3569	0.0293	-0.3511	-470.7278	-546.0020	0.2999	0.6028
0.4442	5.9991	20208	0.8873	0.0047	0.3539	0.0326	-0.3493	-470.7575	-546.0133	0.3014	0.6045

Framework versions

Transformers 4.41.0
Pytorch 2.2.0
Datasets 2.19.1
Tokenizers 0.19.1

dmariko/SmolLM-360M-Instruct-dpo-16k

SmolLM-360M-Instruct-dpo-16k

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

dmariko/
SmolLM-360M-Instruct-dpo-16k