This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: nan
Rewards/chosen: nan
Rewards/rejected: nan
Rewards/accuracies: 0.0
Rewards/margins: nan
Logps/rejected: nan
Logps/chosen: nan
Logits/rejected: nan
Logits/chosen: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.1
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 5
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
8297790832.64	2.3088	100	nan	nan	nan	0.0	nan	nan	nan	nan	nan
0.0	4.6176	200	nan	nan	nan	0.0	nan	nan	nan	nan	nan

Framework versions

Transformers 4.45.0
Pytorch 2.5.1+cu124
Datasets 2.19.2
Tokenizers 0.20.3

stojchet/dpo2

dpo2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

stojchet/
dpo2