This model performs speech emotion recognition (SER) specifically targeting older adults' speech. It is fine-tuned from facebook/hubert-base-ls960 using a two-stage domain-adaptive transfer learning approach, first on concatenated age-independent emotion datasets, then specialised on older adults' speech. Training experiments were conducted using the S3prl toolkit from the SUPERB framework.

Performance

Evaluated on the ElderReact development set:

Metric	Score
Relaxed Precision	0.7333
Relaxed Recall	0.7106
Relaxed F1	0.7218
Jaccard Index	0.6223

Note: Evaluation uses a relaxed matching criterion where a prediction is considered correct if it matches any of the ground-truth labels. This is due to the multi-label nature of the ElderReact dataset, where multiple emotions can be assigned to a single utterance

Emotion Classes

Anger
Happiness
Sadness
Neutral

Training Datasets

This model was trained using the following datasets:

ElderReact — speech recordings from older adults, used for domain-adaptive fine-tuning
RAVDESS — Ryerson Audio-Visual Database of Emotional Speech and Song
IEMOCAP — Interactive Emotional Dyadic Motion Capture Database
CREMA-D — Crowd-sourced Emotional Multimodal Actors Dataset
TESS — Toronto Emotional Speech Set

We thank the creators of all datasets used in this work.

Intended Use

This model is intended for use in conversational AI systems to support empathetic interactions with older adults. It is not intended for clinical or diagnostic purposes.

Usage

Install the required libraries:

pip install transformers torch librosa

For GPU support, install the appropriate version of PyTorch for your CUDA version from pytorch.org

python

from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor
import torch
import librosa

# Load model and feature extractor
model = HubertForSequenceClassification.from_pretrained(
    "NadeeshaP/older-adult-speech-emotion-hubert"
)
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(
    "NadeeshaP/older-adult-speech-emotion-hubert"
)

model.eval()

# Load and preprocess audio (must be 16kHz)
audio, sr = librosa.load("your_audio_file.wav", sr=16000, mono=True)

# Extract features
inputs = feature_extractor(
    audio, 
    sampling_rate=16000, 
    return_tensors="pt", 
    padding=True
)

# Run inference
with torch.no_grad():
    logits = model(**inputs).logits

# Get predicted emotion
predicted_id = torch.argmax(logits, dim=-1).item()
labels = ["Anger", "Happiness", "Sadness", "Neutral"]
print(f"Predicted emotion: {labels[predicted_id]}")

Citation

If you find our project helpful and decide to use this model in your research, please cite our work:

bibtex

@inproceedings{pathirana2026emotion,
    title     = {Emotion Recognition in Older Adults' Speech: Toward Empathetic Conversational AI in Ambient Assisted Living},
    author    = {N. Pathirana and A. Htait and E. Wanner}, 
    booktitle = {International Conference on AI in Healthcare},  
    publisher = {Springer},  
    year      = {2026},  
    note      = {in press}
}

Thank you!