distilgpt_new3_0065 – AI Model by bigmorning | AlphaNeural AI

You can deploy this model and start earning money today!

bigmorning/
distilgpt_new3_0065

like

0

text-generation

generated_from_keras_callback

autotrain_compatible

endpoints_compatible

Views

No views yet

distilgpt_new3_0065

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 2.4971
Validation Loss: 2.3779
Epoch: 64

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.5407	2.4254	0
2.5399	2.4247	1
2.5391	2.4238	2
2.5383	2.4232	3
2.5375	2.4210	4
2.5368	2.4210	5
2.5361	2.4197	6
2.5353	2.4193	7
2.5345	2.4191	8
2.5339	2.4177	9
2.5332	2.4188	10
2.5324	2.4160	11
2.5317	2.4164	12
2.5309	2.4145	13
2.5302	2.4153	14
2.5295	2.4139	15
2.5288	2.4134	16
2.5282	2.4123	17
2.5274	2.4116	18
2.5267	2.4110	19
2.5259	2.4106	20
2.5251	2.4097	21
2.5244	2.4074	22
2.5238	2.4078	23
2.5232	2.4072	24
2.5223	2.4062	25
2.5217	2.4054	26
2.5211	2.4057	27
2.5204	2.4044	28
2.5197	2.4026	29
2.5189	2.4017	30
2.5182	2.4026	31
2.5176	2.4012	32
2.5168	2.4013	33
2.5161	2.3990	34
2.5154	2.3999	35
2.5149	2.3978	36
2.5142	2.3981	37
2.5135	2.3981	38
2.5130	2.3972	39
2.5123	2.3957	40
2.5116	2.3940	41
2.5108	2.3933	42
2.5103	2.3927	43
2.5095	2.3923	44
2.5090	2.3918	45
2.5083	2.3914	46
2.5078	2.3905	47
2.5070	2.3888	48
2.5062	2.3894	49
2.5058	2.3898	50
2.5051	2.3868	51
2.5045	2.3873	52
2.5041	2.3872	53
2.5035	2.3859	54
2.5027	2.3850	55
2.5020	2.3851	56
2.5016	2.3833	57
2.5009	2.3816	58
2.5002	2.3821	59
2.4995	2.3813	60
2.4990	2.3803	61
2.4984	2.3794	62
2.4977	2.3798	63
2.4971	2.3779	64

Framework versions

Transformers 4.20.1
TensorFlow 2.8.2
Datasets 2.3.2
Tokenizers 0.12.1