mt5-small-finetuned-b8-e10-1024-128 – AI Model by sharkMeow | AlphaNeural AI

You can deploy this model and start earning money today!

sharkMeow/
mt5-small-finetuned-b8-e10-1024-128

like

0

text2text-generation

generated_from_trainer

google/mt5-small

endpoints_compatible

Views

No views yet

mt5-small-finetuned-b8-e10-1024-128

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.3822
Rouge1: 13.327
Rouge2: 4.8244
Rougel: 13.1978
Rougelsum: 13.2133
Gen Len: 17.5592

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
4.7372	1.0	1357	3.8287	9.3951	3.6576	9.342	9.3047	12.6653
4.3162	2.0	2714	3.6750	10.9224	4.1119	10.8209	10.8235	15.0997
4.1726	3.0	4071	3.5668	11.7438	4.2353	11.6204	11.6087	16.5169
4.0439	4.0	5428	3.5002	12.402	4.4267	12.2785	12.2924	17.0402
3.9978	5.0	6785	3.4494	12.7762	4.5509	12.6699	12.6829	17.2466
3.9687	6.0	8142	3.4229	12.9652	4.6727	12.8555	12.8761	17.4303
3.8639	7.0	9499	3.4058	13.4216	4.784	13.3097	13.2988	17.4252
3.8474	8.0	10856	3.3924	13.2422	4.7672	13.1416	13.12	17.5046
3.843	9.0	12213	3.3845	13.2519	4.8713	13.1421	13.1304	17.5371
3.8545	10.0	13570	3.3822	13.327	4.8244	13.1978	13.2133	17.5592

Framework versions

Transformers 4.33.0
Pytorch 2.0.0
Datasets 2.14.6
Tokenizers 0.13.3