The model was fine-tuned using Parameter-Efficient Fine-Tuning (LoRA) and the adapter weights were merged into the base model to produce a standalone checkpoint for standard inference.

🔎 Model Overview

Base Model: google/flan-t5-base
Task: Dialogue Summarization
Training Strategy: LoRA (PEFT)
Final Checkpoint: Merged (no adapter required at inference)
Language: English
Evaluation Metric: ROUGE

This model takes multi-turn conversational dialogue as input and generates a concise abstractive summary.

Dataset

The model was fine-tuned on the DialogSum dataset, a benchmark dataset for dialogue summarization containing multi-speaker conversations paired with human-written summaries.

Training Details

Framework: Hugging Face Transformers + PEFT
Backend: PyTorch
Training Environment: Jupyter Notebook
LoRA adapters merged into base model after training

This project demonstrates:

Parameter-efficient fine-tuning
Adapter merging for simplified deployment
End-to-end experimentation and evaluation workflow

Evaluation

The model was evaluated using ROUGE metrics:

Metric	Description
ROUGE-1	Unigram overlap
ROUGE-2	Bigram overlap
ROUGE-L	Longest common subsequence

Example format:

Metric	Score
ROUGE-1	0.484
ROUGE-2	0.236
ROUGE-L	0.399

Usage

Since LoRA adapters have been merged, this model behaves like a standard Hugging Face Seq2Seq model.

Installation

pip install transformers torch

Inference Example

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "sanchitvp/flan-t5-base-finetuned-dialogsum"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

dialogue = """
Person A: Hey, are we still meeting tomorrow?
Person B: Yes, at 10 AM. Don't forget the documents.
Person A: Got it. See you then.
"""

input_text = "Summarize the following dialogue:\n" + dialogue

inputs = tokenizer(
    input_text,
    return_tensors="pt",
    truncation=True,
)

outputs = model.generate(
    **inputs,
    max_new_tokens=64,
)

summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(summary)

Intended Use

This model is intended for:

Dialogue summarization research
NLP experimentation
Educational demonstrations
Portfolio and applied ML showcase

Limitations

Trained specifically on DialogSum
May not generalize well to extremely long or domain-specific conversations
No safety fine-tuning applied
May hallucinate or omit important details

License

This model inherits the Apache-2.0 license from the base model (google/flan-t5-base).

Author

Fine-tuned and deployed by Sanchit Pathak
Built as part of an applied NLP portfolio project.