WavLM-large fine-tuned end-to-end for dimensional emotion regression on naturalistic speech: audio in → continuous Valence / Arousal / Dominance out. Built as the culmination of a two-phase solo project whose full record (including the handcrafted-feature phase this model made obsolete) is at github.com/Nikhiljangra07/voice-emotion-engine.

Headline result

Evaluated on MSP-Podcast v2.0 Test1: ~46k held-out clips from speakers never seen in training (speaker-independent), metric = Concordance Correlation Coefficient:

dimension	CCC
Valence	0.705
Arousal	0.714
Dominance	0.626
mean	0.681

The point of this model is valence: the handcrafted-acoustic-feature literature treats valence as nearly unrecoverable from audio alone. This project measured that ceiling directly — a handcrafted pipeline built first reached CCC 0.06, then 0.35 after diagnosis — and broke it by fine-tuning the self-supervised representation end-to-end: 0.06 → 0.35 → 0.705. The lever was the representation, not the features.

What's in this repo

File	Contents
`model.safetensors` + `config.json`	Fine-tuned WavLM-large backbone (`WavLMModel`)
`head.pt`	Regression head (mean-pool → 256-proj → 3 outputs), PyTorch state dict
`train.log`	Full training log — 3 epochs, per-epoch dev CCC, final Test1 numbers

Usage

python

import torch, torchaudio
from transformers import WavLMModel

backbone = WavLMModel.from_pretrained("Nikhil0097/wavlm-large-emotion-vad")
head = torch.load("head.pt")  # download from this repo; see GitHub for the exact head module
# 16 kHz mono in → backbone hidden states → mean-pool → head → [valence, arousal, dominance]

The exact head module, preprocessing, and a ready CLI (audio file in → VAD JSON out) are in the GitHub repo, which uses this checkpoint as its inference model.

Training data & license

Trained on MSP-Podcast v2.0 (Lotfian & Busso; Busso et al. 2025, arXiv:2509.09791) under its academic license. The dataset is not included and cannot be redistributed — request access from UT-Dallas. Because the training data is research-licensed, this model is released CC-BY-NC-SA 4.0 (non-commercial).

Honest scope

Naturalistic speech (podcast-style, English). Acted-corpus performance does not transfer either direction — the project measured this.
Dominance is the weakest axis (0.626), consistent with the literature.
Single-model checkpoint, no ensembling; numbers are from one training run whose full log is in this repo.