You can deploy this model and start earning money today!

yalhessi/
lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens

Name: lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens
Availability: InStock

peft

safetensors

generated_from_trainer

deepseek-ai/deepseek-coder-1.3b-base

adapter

other

Views

No views yet

lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1287

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0008
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 16
total_eval_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 24
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.2313	0.4001	1440	0.2129
0.1926	0.8002	2880	0.1938
0.1715	1.2003	4320	0.1878
0.1679	1.6004	5760	0.1779
0.1624	2.0006	7200	0.1671
0.1563	2.4007	8640	0.1644
0.1522	2.8008	10080	0.1623
0.1441	3.2009	11520	0.1576
0.1416	3.6010	12960	0.1561
0.1421	4.0011	14400	0.1518
0.1367	4.4012	15840	0.1504
0.138	4.8013	17280	0.1547
0.1273	5.2014	18720	0.1523
0.1287	5.6016	20160	0.1533
0.1269	6.0017	21600	0.1473
0.1237	6.4018	23040	0.1434
0.1217	6.8019	24480	0.1391
0.1151	7.2020	25920	0.1376
0.1157	7.6021	27360	0.1507
0.1171	8.0022	28800	0.1366
0.1083	8.4023	30240	0.1360
0.1091	8.8024	31680	0.1368
0.1023	9.2026	33120	0.1328
0.1044	9.6027	34560	0.1312
0.1022	10.0028	36000	0.1277
0.0963	10.4029	37440	0.1254
0.0965	10.8030	38880	0.1243
0.089	11.2031	40320	0.1272
0.0906	11.6032	41760	0.1286
0.0925	12.0033	43200	0.1235
0.0847	12.4034	44640	0.1241
0.0882	12.8036	46080	0.1174
0.0762	13.2037	47520	0.1203
0.0783	13.6038	48960	0.1212
0.0813	14.0039	50400	0.1215
0.073	14.4040	51840	0.1178
0.0746	14.8041	53280	0.1179
0.0649	15.2042	54720	0.1195
0.068	15.6043	56160	0.1146
0.0694	16.0044	57600	0.1125
0.0617	16.4046	59040	0.1114
0.0633	16.8047	60480	0.1148
0.0562	17.2048	61920	0.1125
0.0559	17.6049	63360	0.1142
0.0571	18.0050	64800	0.1113
0.0494	18.4051	66240	0.1141
0.052	18.8052	67680	0.1080
0.0428	19.2053	69120	0.1192
0.0447	19.6054	70560	0.1129
0.0457	20.0056	72000	0.1180
0.0383	20.4057	73440	0.1142
0.0404	20.8058	74880	0.1170
0.0354	21.2059	76320	0.1178
0.0336	21.6060	77760	0.1152
0.0347	22.0061	79200	0.1207
0.0289	22.4062	80640	0.1251
0.0296	22.8063	82080	0.1258
0.025	23.2064	83520	0.1297
0.0263	23.6066	84960	0.1287

Framework versions

PEFT 0.14.0
Transformers 4.47.0
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0

yalhessi/lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens

lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

yalhessi/
lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-24epochs-no-special-tokens