mBART-TextSimp-LT-BatchSize2-lr1e-4 – AI Model by eglkan1 | AlphaNeural AI

You can deploy this model and start earning money today!

eglkan1/
mBART-TextSimp-LT-BatchSize2-lr1e-4

like

0

text2text-generation

generated_from_trainer

facebook/mbart-large-50

endpoints_compatible

Views

No views yet

mBART-TextSimp-LT-BatchSize2-lr1e-4

This model is a fine-tuned version of facebook/mbart-large-50 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.0962
Rouge1: 0.76
Rouge2: 0.6246
Rougel: 0.7508
Sacrebleu: 53.9078
Gen Len: 32.9976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Sacrebleu	Gen Len
0.0639	1.0	418	0.0779	0.7012	0.5432	0.6904	43.0798	32.9976
0.0653	2.0	836	0.0732	0.7197	0.5593	0.7091	44.8483	32.9976
0.0327	3.0	1254	0.0726	0.7319	0.5787	0.7206	47.842	32.9976
0.0168	4.0	1672	0.0782	0.7466	0.6031	0.7371	50.9225	32.9976
0.013	5.0	2090	0.0804	0.7507	0.6077	0.7409	51.8293	32.9976
0.0032	6.0	2508	0.0846	0.7606	0.6237	0.7507	53.5224	32.9976
0.0012	7.0	2926	0.0911	0.7597	0.6263	0.751	54.0182	32.9976
0.0012	8.0	3344	0.0962	0.76	0.6246	0.7508	53.9078	32.9976

Framework versions

Transformers 4.33.0
Pytorch 2.1.2+cu121
Datasets 2.14.4
Tokenizers 0.13.3