longt5_xl_gov_memsum_bp_15 – AI Model by learn3r | AlphaNeural AI

You can deploy this model and start earning money today!

learn3r/
longt5_xl_gov_memsum_bp_15

like

0

text2text-generation

generated_from_trainer

learn3r/gov_report_memsum_bp

endpoints_compatible

Views

No views yet

longt5_xl_gov_memsum_bp_15

This model is a fine-tuned version of /exports/eddie/scratch/s1970716/models/summarization/longt5_xl_gov_memsum_bp_10/checkpoint-1360 on the learn3r/gov_report_memsum_bp dataset. It achieves the following results on the evaluation set:

Loss: 3.2731
Rouge1: 35.319
Rouge2: 11.8288
Rougel: 16.3777
Rougelsum: 33.5357
Gen Len: 1942.3155

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.001
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 64
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant
num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.1366	1.0	272	3.2731	35.319	11.8288	16.3777	33.5357	1942.3155
0.1183	2.0	545	3.3957	37.5265	12.4932	16.743	35.7285	1867.3145
0.1072	3.0	818	3.4308	41.1487	13.4035	17.5783	39.1233	1561.0853
0.0909	4.0	1091	3.6078	41.2814	13.3137	17.9878	39.2664	1429.4604
0.0834	4.99	1360	3.8803	42.0328	13.9186	17.8203	39.9705	1559.3237

Framework versions

Transformers 4.34.1
Pytorch 2.1.0+cu121
Datasets 2.14.5
Tokenizers 0.14.1