You can deploy this model and start earning money today!

rasendr1ya/
deberta-v3-base-emotion-classifier

Name: deberta-v3-base-emotion-classifier
Availability: InStock

transformers

safetensors

emotion-classification

Views

No views yet

Multilabel Emotion Classification Model (DeBERTa-v3-base)

Model Description

This model is fine-tuned DeBERTa-v3-base for multilabel emotion classification. It can predict multiple emotions simultaneously from text with superior performance using disentangled attention mechanisms.

Emotions Detected

amusement, anger, annoyance, caring, confusion, disappointment, disgust, embarrassment, excitement, fear, gratitude, joy, love, sadness

Performance

Macro F1 Score: 0.3913
Training Data: 37164 samples
Validation Data: 9291 samples

Key Features

Disentangled Attention: Separates content and position representations
Enhanced Mask Decoder: Better handling of masked tokens
Relative Position Bias: Improved positional understanding
Multilabel Capability: Simultaneous prediction of multiple emotions

Usage

python

from transformers import AutoTokenizer, AutoModel
import torch

tokenizer = AutoTokenizer.from_pretrained("your-username/emotion-classifier-deberta")
model = AutoModel.from_pretrained("your-username/emotion-classifier-deberta")

# Example usage
text = "I'm so happy and excited about this!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.sigmoid(outputs.logits)

Training Details

Base Model: microsoft/deberta-v3-base
Training Epochs: 2
Learning Rate: 1e-05
Batch Size: 16
Max Length: 128
Memory Optimizations: Gradient accumulation, FP16, gradient checkpointing

Model Architecture

Total Parameters: 183,842,318
Trainable Parameters: 183,842,318

Training Optimizations

Mixed precision training (FP16)
Gradient accumulation for memory efficiency
Gradient checkpointing
Early stopping based on macro F1 score

rasendr1ya/deberta-v3-base-emotion-classifier