The model demonstrated consistent improvement throughout training, with training loss decreasing from approximately 2.2 to 0.0001 and validation loss improving from around 2.2 to 0.17.

Training Step	Training Loss	Validation Loss
25	2.192	2.185
250	1.290	1.266
500	0.540	0.528
1000	0.159	0.289
1500	0.177	0.318
2000	0.000	0.197
2500	0.000	0.185
2800	0.000	0.174

Usage

python

from transformers import AutoModelForAudioClassification, AutoFeatureExtractor
import torch
import torchaudio

# Load model and feature extractor
model_name = "aicinema69/audio-emotion-detector-large"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
model = AutoModelForAudioClassification.from_pretrained(model_name)

# Prepare audio
audio_path = "path/to/your/audio.wav"
waveform, sample_rate = torchaudio.load(audio_path)
waveform = waveform.squeeze().numpy()

# Resample if needed
if sample_rate != feature_extractor.sampling_rate:
    # Add resampling code here if needed
    pass

# Extract features
inputs = feature_extractor(
    waveform, 
    sampling_rate=feature_extractor.sampling_rate,
    return_tensors="pt"
)

# Predict
with torch.no_grad():
    outputs = model(**inputs)
    predictions = outputs.logits.argmax(-1)

# Map prediction to emotion label
id2label = model.config.id2label
predicted_emotion = id2label[predictions.item()]
print(f"Predicted emotion: {predicted_emotion}")

Limitations

The model is trained on specific voice recordings and may not generalize well to all voices, accents, or recording conditions.
Background noise may significantly affect prediction quality.
The model works best with clear, single-speaker audio samples.
Performance may vary based on recording quality, speaker characteristics, and audio length.

Dataset

This model was fine-tuned on the emotion-tts-dataset created by Skit.ai, which contains labeled audio samples across various emotional categories.

License

This model is shared under the same license as the base model facebook/wav2vec2-large-xlsr-53.

Citation

If you use this model in your research or project, please cite:

@misc{audio-emotion-detector,
  author = {aicinema69},
  title = {Audio Emotion Detector Large},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/aicinema69/audio-emotion-detector-large}}
}

As well as the original wav2vec2 model:

@article{conneau2020unsupervised,
  title={Unsupervised Cross-lingual Representation Learning for Speech Recognition},
  author={Conneau, Alexis and Baevski, Alexei and Collobert, Ronan and Mohamed, Abdelrahman and Auli, Michael},
  journal={arXiv preprint arXiv:2006.13979},
  year={2020}
}

And the emotion-tts-dataset:

@misc{skit-ai-emotion-tts-dataset,
  author = {Skit.ai},
  title = {Emotion TTS Dataset},
  year = {2022},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/skit-ai/emotion-tts-dataset}}
}

aicinema69/audio-emotion-detecto-2

Audio Emotion Detector

Model Description

Training Details

Base Model

Training Hyperparameters

Training Performance

Usage

Limitations

Dataset

License

Citation

aicinema69/
audio-emotion-detecto-2