ROMA is a Real-time Omni-Multimodal Assistant designed for unified streaming audio-video understanding. Unlike traditional videoLLMs that only answer after a query, ROMA integrates both Reactive (Question Answering) and Proactive (Event-Driven Alert, Real-Time Narration) capabilities within a single framework.

ROMA introduces a "Speak Head" mechanism to decouple response timing from content generation, allowing it to autonomously decide when to speak based on the continuous audio-visual stream.

Paper: ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding
Project Page: Link
Repository: [Github (Coming Soon)]

Citation

If you find this project useful, please cite:

bibtex

@article{tian2026roma,
  title={ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding},
  author={Tian, Xueyun and Li, Wei and Xu, Bingbing and Dong, Heng and Wang, Yuanzhuo and Shen, Huawei},
  journal={arXiv preprint arXiv:2601.10323},
  year={2026}
}

EurekaTian/ROMA

ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding

Model Summary

Citation

EurekaTian/
ROMA