The model predicts emotional categories from input sentences and can be used for sentiment analysis, emotion detection, and conversational AI systems. version of the BERT base model. It was introduced in this paper. The code for the distillation process can be found here. This model is uncased: it does not make a difference between english and English.

DistilBERT base model (uncased) This model is a distilled version of the BERT base model. It was introduced in this paper. The code for the distillation process can be found here. This model is uncased: it does not make a difference between english and English.

Model Details

Property	Value
Base Model	`google-bert/bert-base-uncased`
Task	Text Classification
Dataset	GoEmotions
Framework	Hugging Face Transformers
Language	English
License	MIT

Dataset

This model is trained on the GoEmotions dataset, a large dataset of human-annotated emotions extracted from Reddit comments.

Dataset Characteristics

58,000+ Reddit comments
27 emotion labels
Multi-label emotion classification

Dataset Link: https://huggingface.co/datasets/google-research-datasets/go_emotions

Installation

pip install transformers torch

Usage 🚀

Using the Hugging Face Pipeline

python

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="mishbahulal/distilbert-base-uncased"
)

classifier("I am extremely happy today!")

Example Output

python

[
  {
    "label": "joy",
    "score": 0.97
  }
]

Manual Inference

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "mishbahulal/distilbert-base-uncased"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I feel amazing today!"

inputs = tokenizer(text, return_tensors="pt")

outputs = model(**inputs)

prediction = torch.argmax(outputs.logits, dim=1)

print(prediction)

Training Hyperparameters

The following hyperparameters were used during training:

Hyperparameter	Value
Learning Rate	2e-05
Train Batch Size	64
Eval Batch Size	64
Epochs	2
Random Seed	42
Optimizer	AdamW Torch Fused
Betas	(0.9, 0.999)
Epsilon	1e-08
Learning Rate Scheduler	Linear

Optimizer configuration:

OptimizerNames.ADAMW_TORCH_FUSED
betas=(0.9,0.999)
epsilon=1e-08
optimizer_args=None

Training Results

(Add evaluation metrics here if available)

Metric	Value
Accuracy	(add result here)

Framework Versions

The model was trained using the following library versions:

Library	Version
Transformers	5.0.0
PyTorch	2.10.0+cu128
Datasets	4.0.0
Tokenizers	0.22.2

Intended Uses

This model can be used for:

Emotion detection
Sentiment analysis
Social media monitoring
Chatbots with emotional awareness
Customer feedback analysis

Limitations ⚠️

Trained only on English language data
May inherit bias from Reddit-based datasets
Performance may vary on domain-specific text (medical, legal, etc.)

Ethical Considerations

Emotion classification models should be used responsibly. They should not be used for psychological diagnosis or sensitive decision-making systems.

Citation

If you use this model, please cite the GoEmotions paper:

@inproceedings{demszky2020goemotions,
title={GoEmotions: A Dataset of Fine-Grained Emotions},
author={Demszky et al.},
year={2020}
}

Model Maintainer

Created by Mishbahul Al Islam

Contributions and feedback are welcome.