You can deploy this model and start earning money today!

harsharajkumar273/
ProphetNet-Large-Summarization

microsoft/prophetnet-large-uncased

adapter

apache-2.0

Views

No views yet

ProphetNet-Large-Summarization

A fine-tuned version of microsoft/prophetnet-large-uncased for summarizing research papers into concise summaries. This is the first stage of a two-step Research Paper Simplifier pipeline.

Model Description

This model takes a section of a research paper as input and generates a plain-language summary. Fine-tuned using LoRA (PEFT) with 4-bit quantization for efficient training.

Pipeline

Research Paper ──► [ProphetNet-Large-Summarization] ──► Summary ──► [ProphetNet-Large-Story-Generation] ──► Story

Training Details

Parameter	Value
Base model	microsoft/prophetnet-large-uncased
Task	Summarization
Max input length	2048 tokens
Max target length	256 tokens
Learning rate	3e-5
Batch size	2
Gradient accumulation steps	4
Warmup steps	1500
Weight decay	0.01
Fine-tuning method	LoRA (r=16, alpha=64, targets: query_proj, value_proj)
Quantization	4-bit NF4 (bitsandbytes)

Usage

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("harsharajkumar273/ProphetNet-Large-Summarization")
model = AutoModelForSeq2SeqLM.from_pretrained("harsharajkumar273/ProphetNet-Large-Summarization")

text = "Your research paper section here..."
word_count = len(text.split())
prompt = f"Summarize this part of the research paper to less than {word_count // 10} words:\n{text}"

inputs = tokenizer(prompt, return_tensors="pt", max_length=2048, truncation=True)
outputs = model.generate(**inputs, max_length=256, num_beams=4)
summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(summary)

Evaluation Metrics

Evaluated using ROUGE and BERTScore on a held-out 10% test split.

harsharajkumar273/ProphetNet-Large-Summarization