codeparrot-ds – AI Model by mirsiss | AlphaNeural AI

You can deploy this model and start earning money today!

mirsiss/
codeparrot-ds

like

0

text-generation

generated_from_trainer

openai-community/gpt2

autotrain_compatible

text-generation-inference

endpoints_compatible

Views

No views yet

codeparrot-ds

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 5.1436

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 256
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 5
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
9.5238	0.2062	10	8.0787
7.1728	0.4124	20	7.0714
6.6796	0.6186	30	6.8378
6.4165	0.8247	40	6.5842
6.1172	1.0309	50	6.3309
5.838	1.2371	60	6.1147
5.6075	1.4433	70	5.9416
5.4355	1.6495	80	5.8080
5.3001	1.8557	90	5.7022
5.1595	2.0619	100	5.6176
5.0118	2.2680	110	5.5504
4.9696	2.4742	120	5.4852
4.8653	2.6804	130	5.4179
4.8474	2.8866	140	5.3556
4.706	3.0928	150	5.3083
4.6329	3.2990	160	5.2737
4.5585	3.5052	170	5.2405
4.5445	3.7113	180	5.2051
4.497	3.9175	190	5.1791
4.4332	4.1237	200	5.1661
4.3867	4.3299	210	5.1535
4.4075	4.5361	220	5.1473
4.4265	4.7423	230	5.1445
4.4294	4.9485	240	5.1436

Framework versions

Transformers 4.44.2
Pytorch 2.4.1+cu121
Datasets 3.0.0
Tokenizers 0.19.1