mBART-TextSimp-LT-BatchSize4-lr1e-4 – AI Model by eglkan1 | AlphaNeural AI

You can deploy this model and start earning money today!

eglkan1/
mBART-TextSimp-LT-BatchSize4-lr1e-4

like

0

text2text-generation

generated_from_trainer

facebook/mbart-large-50

endpoints_compatible

Views

No views yet

mBART-TextSimp-LT-BatchSize4-lr1e-4

This model is a fine-tuned version of facebook/mbart-large-50 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.0810
Rouge1: 0.8026
Rouge2: 0.6711
Rougel: 0.796
Sacrebleu: 55.7975
Gen Len: 33.3938

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Sacrebleu	Gen Len
0.0978	1.0	209	0.0948	0.6734	0.5117	0.664	43.1877	33.3938
0.0617	2.0	418	0.0598	0.7702	0.6222	0.7609	49.5794	33.3938
1.1021	3.0	627	0.8158	0.0161	0.0	0.0162	0.0009	34.3938
0.0471	4.0	836	0.0822	0.6874	0.5335	0.6779	44.9573	33.3938
0.0276	5.0	1045	0.0664	0.7767	0.6339	0.7686	52.2135	33.3938
0.0162	6.0	1254	0.0756	0.7856	0.6452	0.7796	50.4352	33.3938
0.0069	7.0	1463	0.0796	0.7939	0.6586	0.7877	52.9489	33.3938
0.0051	8.0	1672	0.0810	0.8026	0.6711	0.796	55.7975	33.3938

Framework versions

Transformers 4.33.0
Pytorch 2.1.2+cu121
Datasets 2.14.4
Tokenizers 0.13.3