You can deploy this model and start earning money today!

yalhessi/
small-4e4-notypes

peft

safetensors

generated_from_trainer

deepseek-ai/deepseek-coder-6.7b-base

adapter

other

Views

No views yet

small-4e4-notypes

This model is a fine-tuned version of deepseek-ai/deepseek-coder-6.7b-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1523

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0004
train_batch_size: 4
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 2
gradient_accumulation_steps: 2
total_train_batch_size: 16
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 12
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.6709	0.2001	720	0.2314
0.4529	0.4002	1440	0.1955
0.3782	0.6003	2160	0.1787
0.3562	0.8003	2880	0.1688
0.3295	1.0003	3600	0.1622
0.2893	1.2004	4320	0.1593
0.2811	1.4004	5040	0.1499
0.2777	1.6005	5760	0.1494
0.2723	1.8006	6480	0.1460
0.2624	2.0006	7200	0.1401
0.2361	2.2006	7920	0.1419
0.2268	2.4007	8640	0.1354
0.2224	2.6008	9360	0.1369
0.2224	2.8009	10080	0.1296
0.2249	3.0008	10800	0.1293
0.1885	3.2009	11520	0.1318
0.1923	3.4010	12240	0.1234
0.1888	3.6011	12960	0.1233
0.189	3.8012	13680	0.1233
0.1891	4.0011	14400	0.1236
0.1609	4.2012	15120	0.1267
0.1619	4.4013	15840	0.1250
0.1614	4.6014	16560	0.1257
0.1646	4.8014	17280	0.1196
0.1589	5.0014	18000	0.1205
0.137	5.2015	18720	0.1222
0.137	5.4016	19440	0.1202
0.1415	5.6016	20160	0.1220
0.1416	5.8017	20880	0.1178
0.1377	6.0017	21600	0.1177
0.1195	6.2018	22320	0.1219
0.1198	6.4018	23040	0.1189
0.1197	6.6019	23760	0.1178
0.12	6.8020	24480	0.1200
0.1203	7.0019	25200	0.1214
0.1029	7.2020	25920	0.1242
0.1004	7.4021	26640	0.1207
0.1023	7.6022	27360	0.1225
0.1019	7.8023	28080	0.1156
0.1031	8.0022	28800	0.1227
0.0844	8.2023	29520	0.1235
0.083	8.4024	30240	0.1247
0.0843	8.6025	30960	0.1249
0.0858	8.8026	31680	0.1257
0.0859	9.0025	32400	0.1237
0.0689	9.2026	33120	0.1343
0.0711	9.4027	33840	0.1329
0.0714	9.6028	34560	0.1370
0.0733	9.8028	35280	0.1346
0.0703	10.0028	36000	0.1347
0.0599	10.2029	36720	0.1408
0.0586	10.4029	37440	0.1394
0.0599	10.6030	38160	0.1418
0.0594	10.8031	38880	0.1389
0.0608	11.0031	39600	0.1431
0.052	11.2031	40320	0.1521
0.0516	11.4032	41040	0.1529
0.0516	11.6033	41760	0.1560
0.0515	11.8034	42480	0.1523

Framework versions

PEFT 0.14.0
Transformers 4.47.0
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.1

yalhessi/small-4e4-notypes

small-4e4-notypes

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

yalhessi/
small-4e4-notypes