Official checkpoint for the ICLR 2026 paper - AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization

Emotion understanding is essential for building socially intelligent agents. Although recent multimodal large language models (MLLMs) have shown strong performance on this task, two key challenges remain: (i) spurious associations between emotions and irrelevant audiovisual cues (reasoning errors) and (ii) hallucination of audiovisual cues (perception errors) driven by text priors in the language model backbone.

To quantify and understand these issues, we introduce EmoReAlM, a benchmark designed to evaluate MLLMs for cue–emotion associations, hallucinations, and modality agreement. We then propose AVEm-DPO, a preference optimization technique that aligns model responses with both audiovisual inputs and emotion-centric queries. Specifically, we construct preferences over (i) responses exhibiting spurious associations or hallucinations and (ii) audiovisual input pairs guided by textual prompts. We also include a regularization term that penalizes reliance on text priors, thereby mitigating modality-specific cue hallucinations.

Experimental results on DFEW, RAVDESS, and EMER demonstrate that our method significantly improves the performance of reference baseline models (6–19% relative improvement) in zero-shot settings.

Detailed results and the EmoReAlM leaderboard are available on the project website: avere-iclr.github.io

Please visit our GitHub repository - ihp-lab/AVERE for detailed instructions on how to use the model checkpoints for inference.

This codebase is distributed under the USC Research License. See LICENSE.rst for details.

Portions of this codebase are derived from Vista-DPO and VideoLLaVA; those portions inherit their respective licenses.

AVERE builds upon the following excellent open-source works:

We gratefully acknowledge their contributions to the open-source community.

If you find AVERE or EmoReAlM useful in your research, please cite: