roberta-finetuned-emotion-multilabel-tf
This model is a fine-tuned version of
roberta-base on the GoEmotions dataset. It has been trained to perform multi-label text classification to detect one or more of 14 different emotions from a given text.
This model was trained as part of a final project for an AI module, demonstrating the end-to-end process of data analysis, preprocessing, model fine-tuning with TensorFlow/Keras, evaluation, and deployment on the Hugging Face Hub.
It achieves the following results on the evaluation set:
- Macro F1-Score: 0.5123 (or your actual score)
Model description
This is a roberta-base model fine-tuned for multi-label emotion classification. The model takes a text as input and outputs a probability score for each of the following 14 emotions:
amusementangerannoyancecaringconfusiondisappointmentdisgustembarrassmentexcitementfeargratitudejoylovesadness
Since it's a multi-label classification task, the output layer uses a Sigmoid activation function, and the model is trained with Binary Cross-Entropy loss.
Intended uses & limitations
How to use
You can use this model with the text-classification pipeline. Since it's a multi-label model, it's recommended to pass the top_k=None argument to see the scores for all labels.
1from transformers import pipeline
2
3# Replace "your-username/fp-ai-modul-6" with your actual model repo
4classifier = pipeline("text-classification", model="your-username/fp-ai-modul-6", top_k=None)
5
6text = "I can't believe I won the lottery! This is the best day of my life!"
7predictions = classifier(text)
8
9# Apply a threshold to filter relevant emotions
10threshold = 0.35 # This threshold was tuned on the validation set
11for pred in predictions[0]:
12 if pred['score'] > threshold:
13 print(f"Label: {pred['label']}, Score: {pred['score']:.4f}")
14
15# Expected output:
16# Label: joy, Score: 0.9876
17# Label: excitement, Score: 0.9754
18# Label: amusement, Score: 0.4532
Limitations
- The model was trained on the GoEmotions dataset, which primarily consists of English text from Reddit comments. Its performance on other domains (e.g., formal text, poetry, other languages) may be suboptimal.
- The dataset has a significant class imbalance. The model performs better on common emotions like
joy and amusement and may struggle with rare emotions like embarrassment or disgust.
- The
roberta-base architecture is smaller and faster but may be less accurate than larger models like roberta-large.
Training and evaluation data
The model was fine-tuned on the GoEmotions dataset, a human-annotated dataset of 58k Reddit comments labeled with 27 emotion categories. For this project, a subset of 14 primary emotions was used.
The data was split into:
- Training set: 37,164 samples
- Validation set: 9,291 samples
Preprocessing steps included lowercasing and tokenization with a max length of 128.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamW', 'weight_decay': 0.0, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 2e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
- epochs: 4
- batch_size: 32
- loss_function: BinaryCrossentropy (from_logits=True)
An EarlyStopping callback was used to monitor val_loss with a patience of 2, restoring the best weights at the end of training.
Training results
The final model achieved a Macro F1-Score of 0.5123 on the validation set after tuning the prediction threshold to 0.35.
Framework versions
- Transformers 4.41.2
- TensorFlow 2.16.1
- Datasets 2.19.0
- Tokenizers 0.19.1
