This model is a fine-tuned version of sshleifer/distilbart-cnn-12-6 on the arxiv_summarization_dataset dataset. It achieves the following results on the evaluation set:

Loss: 2.3793
Rouge1: 43.6107
Rouge2: 15.4482
Rougel: 25.4843
Rougelsum: 38.4962
Gen Len: 122.2845

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.6031	1.0	2500	2.4896	42.8688	15.0682	25.1782	37.8758	121.6685
2.3931	2.0	5000	2.4135	43.7802	15.6074	25.7201	38.6496	123.614
2.2454	3.0	7500	2.3819	44.1347	15.8906	25.8923	38.961	120.8765
2.1393	4.0	10000	2.3780	43.6521	15.4051	25.5124	38.3483	122.396
2.0688	5.0	12500	2.3793	43.6107	15.4482	25.4843	38.4962	122.2845

Framework versions

Transformers 4.30.2
Pytorch 2.0.0
Datasets 2.1.0
Tokenizers 0.13.3

mridul3301/distilbart-cnn-12-6-finetuned-arxiv-summarization-20k-5epochs

distilbart-cnn-12-6-finetuned-arxiv-summarization-20k-5epochs

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

mridul3301/
distilbart-cnn-12-6-finetuned-arxiv-summarization-20k-5epochs