You can deploy this model and start earning money today!

angietd/
iemocap-wavlm-large

Name: iemocap-wavlm-large
Availability: InStock

Views

No views yet

Speech Emotion Recognition (IEMOCAP)

UPC Postgrau AI project — Angelica · Andrea · Aitana · Diego.

Detects the speaker's emotion from voice using a fine-tuned WavLM-large model (attention-statistics pooling + MLP head) trained on IEMOCAP. 4 classes — negative (angry/frustrated), positive (happy/excited), sad, neutral — test macro-F1 ≈ 0.60 (best 0.612), accuracy ≈ 0.63, speaker-independent.

Input modes

Upload an audio file (wav/mp3/m4a/flac/ogg) — most reliable.
Record clip from your mic (st.audio_input).
Live mic — streaming via streamlit-webrtc (experimental; WavLM-large on CPU lags — a GPU Space is smoother).
YouTube — paste a URL, audio fetched with yt-dlp (experimental; HF Spaces often rate-limit YouTube).

Adding the trained weights (required for real predictions)

The model file wavlm-large_ft_seed42.pt is ~1.26 GB, so it lives on the Hugging Face Model Hub, not in this Space repo. The app runs with an untrained placeholder until weights are provided.

Step 1 — push the weights to a model repo (run once, in Colab where the file is):

python

from huggingface_hub import HfApi, create_repo
create_repo("YOUR_USERNAME/iemocap-wavlm-large", repo_type="model", exist_ok=True)
HfApi().upload_file(
    path_or_fileobj="/content/drive/MyDrive/ser_cache/wavlm-large_ft_seed42.pt",
    path_in_repo="wavlm-large_ft_seed42.pt",
    repo_id="YOUR_USERNAME/iemocap-wavlm-large",
    repo_type="model",
)

Step 2 — point the Space at it: in the Space → Settings → Variables and secrets, add MODEL_REPO = YOUR_USERNAME/iemocap-wavlm-large (and MODEL_FILE if you renamed the file). Restart the Space; it downloads the weights once on boot and caches them.

(Alternative: commit wavlm-large_ft_seed42.pt into the Space with Git LFS and skip MODEL_REPO — but the Hub route keeps the Space repo light.)

Notes

Domain shift: the model was trained on acted, studio-quality IEMOCAP speech. Real-world audio (laptop mic, YouTube, meetings) will usually score lower than the 0.60 benchmark.
CPU speed: WavLM-large takes ~2–5 s per 4.8 s window on CPU. Long clips are subsampled to stay responsive; for snappy live-mic, run the Space on a T4 GPU.
The architecture in ser_model.FTModel is reproduced verbatim from the v6 notebook so the saved state_dict loads cleanly. The backbone is built from config (no redundant 1.26 GB download).

Repo layout (Docker → Streamlit Space)

Dockerfile            # installs ffmpeg + deps, runs src/streamlit_app.py on :8501
requirements.txt      # python deps (torch, transformers, …)
README.md             # this file (sdk: docker, app_port: 8501)
src/
  streamlit_app.py    # UI
  ser_model.py        # WavLM model + preprocessing + inference

Run locally

bash

pip install -r requirements.txt
export MODEL_REPO=YOUR_USERNAME/iemocap-wavlm-large   # or place the .pt next to ser_model.py
streamlit run src/streamlit_app.py
# or mirror the Space exactly:
docker build -t iemocap-ser . && docker run -p 8501:8501 -e MODEL_REPO=... iemocap-ser

angietd/iemocap-wavlm-large