This model is a fine-tuned version of facebook/bart-large-cnn on the scientific_papers dataset. It achieves the following results on the evaluation set:

Loss: 2.6456
Rouge1: 33.8477
Rouge2: 11.8866
Rougel: 20.1038
Rougelsum: 30.5011

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5.6e-05
train_batch_size: 9
eval_batch_size: 9
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.3695	1.0	56	2.8464	32.1056	10.3835	18.7541	29.2623
2.7639	2.0	112	2.6667	31.2657	10.758	18.9862	28.3279
2.5169	3.0	168	2.6219	33.226	11.4766	19.5923	30.0664
2.2985	4.0	224	2.6029	32.8562	11.5606	19.8616	29.7606
2.0851	5.0	280	2.6456	33.8477	11.8866	20.1038	30.5011

Framework versions

Transformers 4.41.2
Pytorch 2.3.0+cu121
Datasets 2.19.2
Tokenizers 0.19.1

juliosocher/bart-large-cnn-finetuned-scientific-articles

bart-large-cnn-finetuned-scientific-articles

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

juliosocher/
bart-large-cnn-finetuned-scientific-articles