Multimodal Emotion Analyzer for 26 emotions
Model Description
This is a multimodal emotion analysis model that processes both audio and text inputs to predict 26 different emotions with regression scores (0-1).
Architecture
- Text Encoder: RoBERTa-base
- Audio Encoder: CNN-based encoder with 3 convolutional layers
- Fusion: MLP layers with ReLU activation and Dropout
- Output: 26 emotion regression scores (0-1 range)
Emotion Labels
admiration, aesthetic appreciation, awe, anxiety, fear, horror, disgust, calmness, romantic love, sexual desire, nostalgia, interest, surprise, excitement, anger, pride, triumph, contempt, disappointment, empathic pain, sadness, guilt, envy, amusement, awkwardness, adoration
Model Files
best_model.pth: Trained model weights
config.json: Model configuration
training_history.png: Training progress visualization
Usage Example
1import torch
2from transformers import AutoTokenizer
3from models.multimodal_emotion_all import MultimodalEmotionAnalyzerAll
4
5# Load model
6model = MultimodalEmotionAnalyzerAll(num_emotions=26)
7checkpoint = torch.load('best_model.pth', map_location='cpu')
8model.load_state_dict(checkpoint['model_state_dict'])
9model.eval()
10
11# Load tokenizer
12tokenizer = AutoTokenizer.from_pretrained("roberta-base")
13
14# Process text
15text = "I am feeling happy today"
16inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
17
18# Get predictions
19with torch.no_grad():
20 # Note: This model expects both text and audio inputs
21 # For text-only inference, you may need to modify the forward pass
22 outputs = model(**inputs)
23 emotion_scores = torch.sigmoid(outputs)
Training Details
- Training Date: 2025-08-26
- Number of Emotions: 26
- Model Type: Multimodal (Text + Audio)
- Base Model: RoBERTa-base
- Loss Function: MSE Loss
- Optimizer: AdamW
Performance
- Model Size: 1432.7 MB
- Parameters: 125,172,565
Citation
If you use this model in your research, please cite:
@misc{emotion_analyzer_2024,
title={Multimodal Emotion Analyzer},
author={Your Name},
year={2024},
url={
https://huggingface.co/CURI-AI/sentiment-analysis-en-v1}
}