emotion-classify
This model is a fine-tuned version of
bert-base-uncased on the
mteb/emotion dataset.
It is trained to classify English text into one of six emotion categories:
- sadness
- joy
- love
- anger
- fear
- surprise
It achieves the following results on the evaluation set:
- Loss: 0.1660
- Accuracy: 0.9401
Model description
This model uses a pre-trained BERT encoder (bert-base-uncased) with a new classification head on top.
During fine-tuning, the base model's parameters were frozen to retain its general language understanding while the classifier learned emotion-specific patterns from labeled text.
Intended uses & limitations
This model is intended for:
- Emotion classification of English sentences
- Analyzing user sentiment in social media posts, reviews, or feedback
- Educational and research purposes
Limitations
- May not handle sarcasm, code-switching, or very informal language well.
- Emotions are multi-dimensional; the model outputs only one top label per input.
Training and evaluation data
The model was fine-tuned on the mteb/emotion dataset which includes 3 splits:
Train: 15,956 samples
Validation: 1,988 samples
Test: 1,986 samples
Each sample contains:
text: The input sentence
label: An integer (0–5)
label_text: Emotion name
Training procedure
The model was fine-tuned by adding a classification layer on top of the pre-trained BERT base. The SST-2 sentiment dataset was tokenized and padded. Training was done using the Hugging Face Trainer for 3 epochs with evaluation after each epoch. Accuracy was used as the evaluation metric. Finally, the model was tested on a separate test set.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|
| 0.5796 | 1.0 | 998 | 0.1862 | 0.9321 |
| 0.142 | 2.0 | 1996 | 0.1386 | 0.9401 |
| 0.0895 | 3.0 | 2994 | 0.1660 | 0.9401 |
Framework versions
- Transformers 4.52.4
- Pytorch 2.6.0+cu124
- Datasets 3.6.0
- Tokenizers 0.21.2
