t5-small-finetuned-wikihow_3epoch_b8_lr3e-3 – AI Model by Chikashi | AlphaNeural AI

You can deploy this model and start earning money today!

Chikashi/
t5-small-finetuned-wikihow_3epoch_b8_lr3e-3

like

0

text2text-generation

generated_from_trainer

text-generation-inference

endpoints_compatible

Views

No views yet

t5-small-finetuned-wikihow_3epoch_b8_lr3e-3

This model is a fine-tuned version of t5-small on the wikihow dataset. It achieves the following results on the evaluation set:

Loss: 2.3163
Rouge1: 27.1711
Rouge2: 10.6296
Rougel: 23.206
Rougelsum: 26.4801
Gen Len: 18.5433

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.0734	0.25	5000	2.7884	22.4825	7.2492	19.243	21.9167	18.0616
2.9201	0.51	10000	2.7089	24.0869	8.0348	20.4814	23.4541	18.5994
2.8403	0.76	15000	2.6390	24.62	8.3776	20.8736	23.9784	18.4676
2.7764	1.02	20000	2.5943	24.1504	8.3933	20.8271	23.5382	18.4078
2.6641	1.27	25000	2.5428	25.6574	9.2371	21.8576	24.9558	18.4249
2.6369	1.53	30000	2.5042	25.5208	9.254	21.6673	24.8589	18.6467
2.6	1.78	35000	2.4637	26.094	9.7003	22.3097	25.4695	18.5065
2.5562	2.03	40000	2.4285	26.5374	9.9222	22.5291	25.8836	18.5553
2.4322	2.29	45000	2.3858	26.939	10.3555	23.0211	26.2834	18.5614
2.4106	2.54	50000	2.3537	26.7423	10.2816	22.7986	26.083	18.5792
2.3731	2.8	55000	2.3163	27.1711	10.6296	23.206	26.4801	18.5433

Framework versions

Transformers 4.18.0
Pytorch 1.10.0+cu111
Datasets 2.0.0
Tokenizers 0.11.6