This model is a fine-tuned version of DistilBERT for emotion classification tasks. It is trained on the dair-ai/emotion dataset, which contains short text samples categorized into six emotions:

Anger
Fear
Joy
Love
Sadness
Surprise

The model uses the lightweight DistilBERT architecture, making it efficient for deployment while maintaining strong performance on text classification tasks.

Limitations & Biases

Generalization: The model may not generalize well to non-English text, noisy inputs, or texts significantly different from the training data. It truncates inputs longer than 128 tokens, which may affect performance on long texts.
Ambiguity: Struggles with mixed emotions, sarcasm, or nuanced contexts. It’s limited to single-label classification for six predefined emotions.
Dataset Bias: Performance may be skewed due to class imbalances or cultural biases in the training dataset (dair-ai/emotion).
Model Bias: Inherits potential biases from the pre-trained DistilBERT, including gender, racial, or socioeconomic biases.

Evaluation & Training

-Training Dataset: dair-ai/emotion (16,000 examples)

Highest Validation Accuracy: 94.05%
Final Validation Loss: 0.158
Test Accuracy 92.3%
Metrics Used:
- Accuracy: Percentage of correctly classified labels in the validation set.
- Loss: Cross-entropy loss on the validation set, used to measure the model's confidence in predictions.

The model achieved a strong performance on the dair-ai/emotion validation dataset, indicating its ability to classify text into six emotion categories effectively.

Training Hyperparameters

The following hyperparameters were used during training:

Learning Rate: 2e-05
Train Batch Size: 16
Eval Batch Size: 32
Seed: 42
Optimizer: adamw_torch with betas=(0.9, 0.999) and epsilon=1e-08
Learning Rate Scheduler Type: Linear
Number of Epochs: 3
Weight Decay: 0.05

Example Usage Code

python

from transformers import pipeline

# Load the fine-tuned model
classifier = pipeline("text-classification", model="moeMachineLearning/emotion-classifier-1.0")

# Test texts
texts = [
    "I am so happy today!",
    "This is the worst day of my life.",
    "I'm feeling a bit nervous about tomorrow's event.",
]

# Run predictions
for text in texts:
    result = classifier(text)
    print(f"Text: {text}")
    print(f"Prediction: {result}\n")

moeMachineLearning/emotions-classifier-1.0

emotions-classifier-1.0

Model Description

Limitations & Biases

Evaluation & Training

Training Hyperparameters

Example Usage Code

moeMachineLearning/
emotions-classifier-1.0