FLAN-T5 Small Fine-Tuned on Gigaword for Headline Generation
This model is a fine-tuned checkpoint of google/flan-t5-small for turning short news text into concise headline-style output.
Training Setup
- Dataset:
SalmanFaroz/gigaword (article -> summary)
- Input format:
headline: <article>
- Output format:
<headline>
- Train samples:
4000
- Validation samples:
300
- Max steps:
70
- Training loss:
2.465707833426339
Usage (Transformers)
1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
2
3model_id = "vtayyab6/flan-t5-small-gigaword-headline-ft"
4tokenizer = AutoTokenizer.from_pretrained(model_id)
5model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
6
7text = "The government announced a new transport reform plan on Tuesday to improve rail connectivity across the country."
8prompt = f"headline: {text}"
9inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=192)
10outputs = model.generate(**inputs, max_new_tokens=20, num_beams=4, do_sample=False)
11print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Sample Predictions
Example 1
Input: in a blow to his attorney 's efforts to win a reduced prison sentence , a court-ordered brain scan revealed no abnormalities that might explain the behavior of lionel tate , once the youngest person in modern u.s. his...
Target: tate brain scan shows no abnormalities
Prediction: lionel tate 's brain scan shows no abnormalities
Example 2
Input: the eldest daughter of former chilean dictator augusto pinochet was arrested at a washington airport after fleeing chile where she has been charged with tax fraud , a government spokesman said wednesday .
Target: pinochet 's daughter arrested in washington
Prediction: eldest daughter of chilean dictator arrested at washington airport
Example 3
Input: a rights group said thursday that bangladesh should be expelled from the u.n. human rights council and be banned from taking part in peacekeeping missions until it addresses abuses by the nation 's police and security...
Target: rights group urges expulsion of bangladesh from un human rights council
Prediction: bangladesh should be expelled from u.n. human rights council
Notes
- This checkpoint was trained as a fast portfolio demo using a subsample.
- For stronger quality, train with larger sample size and more steps.
