gpt2-wikitext2 – AI Model by testingtime | AlphaNeural AI

You can deploy this model and start earning money today!

testingtime/
gpt2-wikitext2

like

0

text-generation

generated_from_trainer

openai-community/gpt2

autotrain_compatible

text-generation-inference

endpoints_compatible

Views

No views yet

gpt2-wikitext2

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 5.1991

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 50000

Training results

Training Loss	Epoch	Step	Validation Loss
6.4398	1.0	2241	6.3596
6.0236	2.0	4482	6.0117
5.6951	3.0	6723	5.7901
5.4821	4.0	8964	5.6291
5.2827	5.0	11205	5.5087
5.1335	6.0	13446	5.4132
4.9653	7.0	15687	5.3436
4.8575	8.0	17928	5.2822
4.7115	9.0	20169	5.2414
4.5957	10.0	22410	5.2125
4.4842	11.0	24651	5.1877
4.3761	12.0	26892	5.1672
4.2879	13.0	29133	5.1585
4.1845	14.0	31374	5.1579
4.1079	15.0	33615	5.1553
4.0398	16.0	35856	5.1583
3.9677	17.0	38097	5.1684
3.8959	18.0	40338	5.1724
3.8508	19.0	42579	5.1800
3.8001	20.0	44820	5.1902
3.7679	21.0	47061	5.1943
3.7323	22.0	49302	5.1982
3.7191	22.3115	50000	5.1991

Framework versions

Transformers 4.44.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1