Model Card for T5 Small Fine-Tuned for Title Generation
This model fine-tunes the T5-small model from Hugging Face for the task of news article title generation. It has been trained on a dataset where each input sequence is a body of text, and the task is to generate a coherent and meaningful title that summarizes the content.
The dataset for Title Generation comprises a collection of articles along with their corresponding titles and subjects. Each entry in the dataset includes the full text of an article, its subject category, and the title that summarizes its content. This batchset serves as a valuable resource for natural language processing tasks, particularly in the domain of text summarization and headline generation. Researchers, developers, and enthusiasts can leverage this batchset to train machine learning models for automatic title generation, headline summarization, and related applications. With a diverse range of subjects and article texts, this batchset offers ample opportunities for experimentation and innovation in the field of natural language processing.
Model Details
Model Description
This fine-tuned model is based on google/t5-small and has been trained to generate titles for textual inputs. The model was fine-tuned using the 🤗 Transformers library, and ROUGE was used as an evaluation metric to assess performance during training.
- Developed by: Ivanhoe9
- Model type: T5 (sequence-to-sequence)
- Language(s) (NLP): English
- License: Apache-2.0
- Finetuned from model: google/t5-small
Model Sources
Uses
Direct Use
The model can be directly used for generating titles from text. It is well-suited for summarizing articles, blogs, or any content that requires a concise title.
Out-of-Scope Use
The model might not perform well on texts that are highly technical, ambiguous, or outside its training domain.
Bias, Risks, and Limitations
This model may generate biased or inappropriate titles if the input text contains sensitive or biased content. Users should validate generated titles before using them in production.
How to Get Started with the Model
You can use the following code to start generating titles:
1from transformers import T5ForConditionalGeneration, T5Tokenizer
2
3tokenizer = T5Tokenizer.from_pretrained("Ivanhoe9/finetune_T5_small_title_generation_NLP_cours")
4model = T5ForConditionalGeneration.from_pretrained("Ivanhoe9/finetune_T5_small_title_generation_NLP_cours")
5
6article = """Neutral Switzerland is in touch with Spain and Catalonia about resolving a row over the region s moves towards
7independence but conditions for promoting talks are not yet ripe, the foreign ministry in Bern said on Friday. Facilitation
8can only be provided if both parties request it. Switzerland is in contact with both parties, but the conditions for facilitation
9are not in place at this stage, a spokesman said by email in response to a query. He described the situation in Catalonia as an internal
10Spanish political matter and said Switzerland respected the sovereignty of Spain."""
11
12input_text = "Generate a title: " + article
13input_ids = tokenizer.encode(input_text, return_tensors="pt")
14
15# Generate title
16generated_ids = model.generate(input_ids, max_length=30, num_beams=4, early_stopping=True)
17title = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
18print(title)
19
Training Details
Training Data
The model was trained on a dataset of texts and their respective titles, preprocessed to include the prefix "Generate a title:" to guide the T5-small architecture. The data was tokenized using Hugging Face's T5 tokenizer.
Training Procedure
Preprocessing
- Input texts were cleaned using a custom
clean_text function.
- A prefix "Generate a title:" was added to guide the model.
Training Hyperparameters
- Batch size: 32
- Number of epochs: 9
- Learning rate: 1e-4
- Optimizer: Adam
- Loss function: Cross-Entropy Loss
Checkpoints and Early Stopping
- Checkpoints were saved based on validation loss improvements.
- Early stopping was used with patience set to avoid overfitting.
Evaluation
Testing Data, Factors & Metrics
Testing Data
Evaluation was performed using a portion of the dataset held out during training.
Metrics
- ROUGE-1 and ROUGE-2 scores were computed during training to assess how well the generated titles captured relevant content from the input text.
Environmental Impact
The carbon footprint and energy consumption can be estimated using tools like the
Machine Learning Impact Calculator.
- Hardware Type: GPU A100 (NVIDIA)
- Hours used: 0.5 H
- Compute Region: GLICID HPC
Technical Specifications
Model Architecture and Objective
- T5-small with a sequence-to-sequence objective.
Citation
If you use this model, please cite the following:
1@misc{ivanhoe2025t5tune,
2 title={Fine-Tuning T5 Small for Title Generation},
3 author={Ivanhoe9},
4 year={2025},
5 url={https://huggingface.co/Ivanhoe9/finetune_T5_small_title_generation_NLP_cours}
6}
