You can deploy this model and start earning money today!

yogkul2000/
AVATAR

Name: AVATAR
Rating: 2.5 (13 reviews)

safetensors

qwen2_5_omni

2508.03100

Views

No views yet

AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video

Yogesh Kulkarni, Pooyan Fazli

Abstract

Multimodal reasoning over long-horizon video is challenging due to the need for precise spatiotemporal fusion and alignment across modalities. While recent methods such as Group Relative Policy Optimization (GRPO) have shown promise in this domain, they suffer from three key limitations: (1) data inefficiency from their on-policy design, (2) a vanishing advantage problem, where identical or near-identical rewards within a group eliminate the learning signal by producing zero-valued advantages, and (3) uniform credit assignment that fails to emphasize critical reasoning steps.

We introduce AVATAR (Audio-Video Agent for Alignment and Reasoning), a framework that addresses these limitations through two core components: (1) an off-policy training architecture that improves sample efficiency and resolves vanishing advantages by reusing past experiences with greater reward diversity, and (2) Temporal Advantage Shaping (TAS), a novel credit assignment strategy that upweights key reasoning phases during learning.

AVATAR achieves strong performance across various benchmarks, outperforming the Qwen2.5-Omni baseline by +5.4 on MMVU, +4.9 on OmniBench, and +4.5 on Video-Holmes, while demonstrating over 35% higher sample efficiency. These results demonstrate that targeted RL improvements, rather than massive architectural changes, effectively address core multimodal reasoning challenges.

📦 Install

Environment Setup

bash

conda create -n avatar python=3.10
conda activate avatar

# Install PyTorch with CUDA 12.6
pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu126

# Install flash attention (if facing issues, use the command below)
pip install flash-attn==2.7.4.post1

# If flash-attn installation fails, try:
pip install flash-attn==2.7.4.post1 --no-build-isolation

pip install transformers==4.54.1

# Install other dependencies
pip install decord opencv-python pillow numpy
pip install qwen-omni-utils[decord] -U

Single Video Inference

python

import os
import torch
from transformers import (
    Qwen2_5OmniThinkerForConditionalGeneration,
    Qwen2_5OmniProcessor,
)
from qwen_omni_utils import process_mm_info


def run_inference():
    MODEL_PATH = ""
    VIDEO_PATH = ""
    QUESTION = "Use available audio and video to answer: Why the person is doing what they are doing? Give reasoning between <think> and </think> tags."

    device = "cuda:0"
    use_audio_flag = True

    print("Loading model...")
    model = Qwen2_5OmniThinkerForConditionalGeneration.from_pretrained(
        MODEL_PATH,
        device_map=device,
        torch_dtype=torch.bfloat16,
        attn_implementation="flash_attention_2",
    ).eval()
    processor = Qwen2_5OmniProcessor.from_pretrained(MODEL_PATH)
    print("Model loaded.")

    content_items = [
        {"type": "video", "video": VIDEO_PATH},
        {"type": "text", "text": QUESTION},
    ]

    conv = [{"role": "user", "content": content_items}]

    prompt_text = processor.apply_chat_template(
        conv, add_generation_prompt=True, tokenize=False
    )

    try:
        audios, images, videos = process_mm_info(
            conv, use_audio_in_video=use_audio_flag
        )
    except Exception as e:
        print(f"Failed to process with audio, retrying without: {e}")
        use_audio_flag = False
        audios, images, videos = process_mm_info(conv, use_audio_in_video=False)

    inputs = processor(
        text=prompt_text,
        audio=audios,
        images=images,
        videos=videos,
        return_tensors="pt",
        padding=True,
        use_audio_in_video=use_audio_flag,
    ).to(device)

    print("Generating response...")
    with torch.no_grad():
        out_ids = model.generate(
            **inputs,
            use_audio_in_video=use_audio_flag,
            do_sample=False,
            max_new_tokens=512,
        )

    reply = processor.batch_decode(out_ids, skip_special_tokens=True)[0]

    print("\n" + "=" * 20 + " MODEL OUTPUT " + "=" * 20)
    print(reply)
    print("=" * 54 + "\n")


if __name__ == "__main__":
    run_inference()

📝 Citation

If you find AVATAR useful for your research, please cite our paper:

bib

@article{kulkarni2025avatar,
  title={AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video},
  author={Kulkarni, Yogesh and Fazli, Pooyan},
  journal={arXiv preprint arXiv:2508.03100},
  year={2025}
}

📪 Contact

For questions about the paper, please contact Yogesh Kulkarni at ykulka10@asu.edu. You can also open an issue in this GitHub repository for bugs or specific questions related to the code.

yogkul2000/AVATAR