Model Card: wavlm-finetune-natural-balance
Model Summary
microsoft/wavlm-base-plus fine-tuned on a realistically distributed Russian-language dataset of emotional speech for the task of 7-class emotion classification. Fine-tuning performed as part of the EchoStressAI project.
This model captures emotionally colored vocal expressions typical of semi-spontaneous speech, preserving the natural imbalance of emotion classes observed in real-world human communication.
Model Details
- Model type:
WavLMForSequenceClassification
- Pretrained base:
microsoft/wavlm-base-plus
- Fine-tuned dataset: Natural distribution of 7 emotions from Dusha and EmoGator datasets
- Languages: Russian
- Task: Speech emotion recognition (SER)
Label Mapping
| ID | Label |
|---|
| 0 | Angry |
| 1 | Disgusted |
| 2 | Happy |
| 3 | Neutral |
| 4 | Sad |
| 5 | Scared |
| 6 | Surprised |
Note:
Disgusted includes samples labeled as Disgust and Contempt.Scared includes Fear, Distress, Embarrassment, and Guilt.
These groupings are based on emotional proximity in affective psychology (Plutchik, Geneva Wheel).
Training Details
- Epochs: 3
- Batch size: 8
- Learning rate: 2e-5
- Optimizer: AdamW
- Scheduler: Linear
- Warmup steps: 500
- Loss function: CrossEntropyLoss
- FP16 training: Enabled
- Evaluation split: Stratified validation (225 samples per class, total 1575)
Evaluation Results (Validation Set, 1575 samples)
| Metric | Value |
|---|
| Accuracy | 0.8597 |
| F1-score (macro avg) | 0.8582 |
| F1-score (weighted) | 0.8582 |
| Precision (macro) | 0.8640 |
| Recall (macro) | 0.8597 |
Per-class F1-scores:
- Angry: 0.8966
- Disgusted: 0.8374
- Happy: 0.8477
- Neutral: 0.7864
- Sad: 0.8481
- Scared: 0.7980
- Surprised: 0.9933
Observations:
- Most confusion occurred between
Neutral ↔ Sad, and Scared ↔ Disgusted, due to acoustic overlap in emotional intensity.
Surprised was classified with very high precision and recall.
Datasets and Licensing
The model was trained using Russian-language emotional speech data from the following open-access sources:
-
Dusha dataset —
CC BY 4.0
Used to represent a range of naturally occurring emotional expressions
-
EmoGator dataset —
Apache 2.0
Used to enrich underrepresented classes and balance valence-arousal coverage
Appropriate attribution and license compliance are maintained in the use of these datasets.
License
This model is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0).
You are free to use, modify, and distribute the model and its outputs — including for commercial purposes — with proper attribution.
Please cite the original datasets (Dusha, EmoGator) in any derivative work.
Intended Use
- Target: Analysis of emotional speech in Russian
- Use cases:
- AI assistants with emotional awareness
- Mental state monitoring systems (e.g., EchoStressAI)
- Human-robot or human-computer interaction research
- Experimental affective computing studies
Limitations
- Model trained on clean, annotated data — performance may degrade on noisy, spontaneous, or dialectal speech.
- Emotion taxonomy is limited to 7 classes.
- Some emotion pairs remain difficult to separate (e.g., Neutral vs Sad).
Citation
To be added after the formal publication of the EchoStressAI project.
Contact
Developed by
https://huggingface.co/nikatonika
Project:
EchoStressAI
