SER-CNN-Transformer

Speech Emotion Recognition — CNN-Transformer + Multidimensional Attention

Highlights

• Hybrid CNN + Transformer backbone for robust speech representation
• Multidimensional Attention Mechanism for improved emotion discrimination
• Scripts for preprocessing, training, and inference on IEMOCAP features

Requirements

Quick Start

1. Preprocess IEMOCAP (optional if using built-in loader):

1. Train a model with MFCC features (example):

1. Run inference on an audio file (feature extraction + predict):

• Use -g to select GPU id (set to -1 for CPU).
• Use -f to change feature type (e.g., mfcc, fbank if supported).

Data

• The repository contains helper scripts in preprocessing/ to prepare the IEMOCAP features.
• Example dataset folder: data/IEMOCAP/ with precomputed feature files and checkpoints.

Project Structure

• train_IEMOCAP.py — training entrypoint
• predict.py — inference script
• models.py — model definitions (CNN-Transformer and attention modules)
• data_loader.py — dataset and dataloader utilities
• preprocessing/ — data processing helpers for IEMOCAP

Pretrained Models & Checkpoints

• Example checkpoint included: data/IEMOCAP/model_CTMAM_mfcc_all.pth and metric/loss logs in the same folder.
• To evaluate or resume training, point --model-path to the checkpoint file.

Evaluation

• Training script saves checkpoints and logs final metrics to data/IEMOCAP/.
• Add evaluation code or use predict.py to compute per-file predictions and aggregate metrics.

License