t5-small-science-papers-NIPS – AI Model by Dagar | AlphaNeural AI

You can deploy this model and start earning money today!

Dagar/
t5-small-science-papers-NIPS

like

0

text2text-generation

generated_from_trainer

text-generation-inference

endpoints_compatible

Views

No views yet

t5-small-science-papers-NIPS

This model is a fine-tuned version of Dagar/t5-small-science-papers on the None dataset. It achieves the following results on the evaluation set:

Loss: 4.7566
Rouge1: 15.7066
Rouge2: 2.5654
Rougel: 11.4679
Rougelsum: 14.4017
Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	318	5.1856	13.7172	2.0644	10.2189	12.838	19.0
5.4522	2.0	636	5.0383	15.6211	2.1808	11.3561	14.3054	19.0
5.4522	3.0	954	4.9486	15.1659	2.3308	11.1052	13.9456	19.0
5.1254	4.0	1272	4.8851	15.716	2.4099	11.4954	14.5099	19.0
4.9794	5.0	1590	4.8456	15.5507	2.4267	11.3867	14.3237	19.0
4.9794	6.0	1908	4.8073	15.8406	2.4254	11.6878	14.6154	19.0
4.8823	7.0	2226	4.7872	15.5554	2.4637	11.3401	14.3183	19.0
4.8338	8.0	2544	4.7680	15.4783	2.4888	11.3364	14.2031	19.0
4.8338	9.0	2862	4.7621	15.958	2.5662	11.6139	14.6576	19.0
4.7838	10.0	3180	4.7566	15.7066	2.5654	11.4679	14.4017	19.0

Framework versions

Transformers 4.24.0
Pytorch 1.12.1+cu113
Datasets 2.7.1
Tokenizers 0.13.2