T5-base Fine-tuned on XSum using Feature Extraction Transfer Learning Technique
This repository contains a PyTorch implementation for fine-tuning the flax-community/t5-base-cnn-dm model on the XSum dataset.
The task is extreme summarization — generating a concise one-sentence summary from a full BBC news article.
Final Results
After 12 epochs of fine-tuning, the model achieved the following ROUGE scores on the validation set:
ROUGE-1: 35.28%
ROUGE-2: 14.18%
ROUGE-L: 28.91%
ROUGE-Lsum: 28.91%
Average training loss after the final epoch: 1.2611
