T5-LoRA-Financial: LoRA Fine-Tuned T5 Model for Financial News Summarization

Model Overview

• Model Architecture: T5-base
• Task: Text summarization (financial news → news title)
• Fine-Tuning Technique: LoRA (Low-Rank Adaptation)
• Pretrained Checkpoint: mrm8488/t5-base-finetuned-summarize-news
• Dataset: ashraq/financial-news-articles (80/20 train/test split)
• Evaluation Metric: ROUGE
• Framework: Hugging Face Transformers + PEFT + PyTorch

LoRA Configuration

• Rank (r): 8
• Alpha (lora_alpha): 64
• Target Modules: q, v (query and value projections in attention)
• Dropout (lora_dropout): 0.05
• Bias: none
• Task Type: Sequence-to-sequence LM

Only LoRA parameters are trained; the base T5 parameters remain frozen.

Evaluation Results

These results are after 15 epochs of LoRA fine-tuning on 4 GPUs.