You can deploy this model and start earning money today!

on1onmangoes/
wav2vec2-emotion-classifier-V2

Name: wav2vec2-emotion-classifier-V2
Availability: InStock

multi-label-classification

custom_emotion_dataset

apache-2.0

model-index

Views

No views yet

Wav2Vec2 Multi-Label Emotion Classifier

This model is a fine-tuned version of facebook/wav2vec2-base for multi-label emotion classification from audio.

Model Description

The model uses a Hydra architecture with multiple binary classification heads, one for each emotion. It supports ignore labels (-100) for handling uncertain or unlabeled data points.

Supported Emotions

angry, disgust, fear, happy, neutral, depress, surprise, frustrated, excited

Training Results

Overall Performance

Best Validation F1 Score: 0.0000
Best Validation Accuracy: 0.0000
Best Validation Loss: inf
Best Epoch: 0

Per-Emotion Performance (at best epoch)

Emotion	Accuracy	Valid Samples	Ignored Samples
angry	0.8762	4749	0
disgust	0.9872	4749	0
fear	0.9821	4749	0
happy	0.8793	4749	0
neutral	0.6477	4749	0
depress	0.9111	4749	0
surprise	0.9299	4749	0
frustrated	0.6949	1996	2753
excited	0.7996	1996	2753

Sample Statistics

Total Training Samples: 18994
Total Validation Samples: 4749
Total Valid Labels: 37235
Total Ignored Labels: 5506
Ignore Rate: 12.9%

Training Configuration

json

{
  "model_id": "facebook/wav2vec2-base",
  "emotion_labels": [
    "angry",
    "disgust",
    "fear",
    "happy",
    "neutral",
    "depress",
    "surprise",
    "frustrated",
    "excited"
  ],
  "num_classes": 9,
  "ignore_label": -100,
  "batch_size": 4,
  "num_epochs": 6,
  "learning_rate": 0.0001,
  "warmup_steps": 100,
  "gradient_accumulation_steps": 4,
  "classification_threshold": 0.5,
  "weight_decay": 0.01,
  "max_grad_norm": 1.0,
  "seed": 42
}

Usage

Handling Ignore Labels

This model was trained with support for ignore labels (-100). During inference:

The model outputs raw logits for each emotion
Apply sigmoid to get probabilities
Use a threshold (default: 0.5) for binary classification
The model can handle inputs where some emotions were marked as "ignore" during training

Example Code

python

import torch
from transformers import Wav2Vec2Processor

# Load model and processor
model = # Load your model here
processor = Wav2Vec2Processor.from_pretrained('mangoesai/wav2vec2-emotion-classifier-V2')

# Process audio
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt", padding=True)

# Get predictions
with torch.no_grad():
    outputs = model(inputs.input_values, inputs.attention_mask)
    probabilities = torch.sigmoid(outputs)
    predictions = (probabilities > 0.5).int()

Training Infrastructure

Hardware: NVIDIA A100-SXM4-40GB
Training Duration: 1.0 hours
PyTorch Version: 2.6.0+cu124
Transformers Version: 4.36.0

Limitations and Biases

The model's performance may vary for emotions with limited training data
Emotions with high ignore rates during training may have reduced accuracy
The model is trained on 16kHz audio and may need resampling for other sample rates

Citation

If you use this model, please cite:

bibtex

@misc{wav2vec2-emotion-classifier,
  author = {Your Name},
  title = {Wav2Vec2 Multi-Label Emotion Classifier},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/mangoesai/wav2vec2-emotion-classifier-V2}
}

on1onmangoes/wav2vec2-emotion-classifier-V2