t5-small-finetuned-booksum – AI Model by mgundik | AlphaNeural AI

You can deploy this model and start earning money today!

mgundik/
t5-small-finetuned-booksum

like

0

text2text-generation

generated_from_trainer

google-t5/t5-small

text-generation-inference

endpoints_compatible

Views

No views yet

t5-small-finetuned-booksum

This model is a fine-tuned version of t5-small on an booksum dataset. It achieves the following results on the evaluation set:

Loss: 3.3115
Rouge1: 20.5085
Rouge2: 2.9908
Rougel: 13.8508
Rougelsum: 18.4822
Gen Len: 228.1577

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 6
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.7585	1.0	600	3.3969	17.6639	2.5697	12.3838	15.9802	307.0236
3.5518	2.0	1200	3.3526	20.0469	2.9053	13.6055	18.0492	248.2581
3.5108	3.0	1800	3.3318	20.0243	2.8879	13.5558	17.9889	245.3416
3.4798	4.0	2400	3.3202	20.1501	2.9346	13.6819	18.1977	232.3801
3.4542	5.0	3000	3.3134	20.6061	3.0311	13.9844	18.5832	217.8302
3.453	6.0	3600	3.3115	20.5085	2.9908	13.8508	18.4822	228.1577

Framework versions

Transformers 4.42.4
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1