t5-small-finetuned-pubmed – AI Model by andkelly21 | AlphaNeural AI

You can deploy this model and start earning money today!

andkelly21/
t5-small-finetuned-pubmed

like

0

text2text-generation

generated_from_trainer

text-generation-inference

endpoints_compatible

Views

No views yet

t5-small-finetuned-pubmed

This model is a fine-tuned version of t5-small on a truncated PubMed Summarization dataset. It achieves the following results on the evaluation set:

Loss: 2.7252
Rouge1: 19.4457
Rouge2: 3.125
Rougel: 18.3168
Rougelsum: 18.5625

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5.6e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.2735	1.0	13	2.9820	18.745	3.7918	15.7876	15.8512
3.0428	2.0	26	2.8828	17.953	2.5	15.49	15.468
2.6259	3.0	39	2.8283	21.5532	5.9278	19.7523	19.9232
3.0795	4.0	52	2.7910	20.9244	5.9278	19.8685	20.0181
2.8276	5.0	65	2.7613	20.6403	3.125	18.0574	18.2227
2.64	6.0	78	2.7404	19.4457	3.125	18.3168	18.5625
2.5525	7.0	91	2.7286	19.4457	3.125	18.3168	18.5625
2.4951	8.0	104	2.7252	19.4457	3.125	18.3168	18.5625

Framework versions

Transformers 4.25.1
Pytorch 1.13.0
Datasets 2.8.0
Tokenizers 0.13.2