📈 Evaluation

Developed by: Prasanna Nagarale
Model type: Causal Language Model (Decoder-only Transformer)
Base model: GPT-2
Fine-tuning method: QLoRA (PEFT)
Task: Text Summarization
Framework: Hugging Face Transformers + PEFT
Language: English

📊 Metrics Used

ROUGE-1, ROUGE-2, ROUGE-L
Used to evaluate summary quality

📉 Results (Baseline vs Fine-Tuned)

Model	ROUGE-1	ROUGE-2	ROUGE-L
GPT-2 (baseline)	~0.16	~0.09	~0.12
Phi-2 (baseline)	~0.17	~0.096	~0.13

👉 Fine-tuned model shows improved contextual summarization capability.

🚀 Usage

🔹 Load Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("prasanna030/gpt2-qlora-summarizer")
tokenizer = AutoTokenizer.from_pretrained("prasanna030/gpt2-qlora-summarizer")

💡 Intended Use
News article summarization
Content condensation
Educational demos for fine-tuning LLMs
Low-resource NLP experimentation

⚠️ Limitations
GPT-2 is not inherently optimized for summarization
May generate:
repetitive text
incomplete summaries
Performance limited due to:
small dataset subset
lightweight training

🚫 Out-of-Scope Use
Not suitable for:
critical decision-making
medical/legal summarization
factual verification tasks

🧠 Key Insight

This project demonstrates that:

Parameter-efficient fine-tuning methods like QLoRA can significantly improve model performance even on limited hardware.

📦 Training Environment
Platform: Google Colab
GPU: T4 
Libraries:
transformers
peft
datasets
bitsandbytes

📜 License

This model follows the license of the base model (GPT-2).

🙌 Acknowledgements
Hugging Face 🤗
CNN/DailyMail Dataset
Open-source LLM community

📬 Contact

Developed by Prasanna Nagarale