RoboPoint is an open-source vision-language model instruction-tuned on a mix of robotics and VQA data. Given an image with language instructions, it outputs precise action guidance as points.
Primary Use Cases
RoboPoint can predict spatial affordances—where actions should be taken in relation to other entities—based on instructions. For example, it can identify free space on a shelf in front of the rightmost object.
If you find our work helpful, please consider citing our paper.
@article{yuan2024robopoint,
title={RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics},
author={Yuan, Wentao and Duan, Jiafei and Blukis, Valts and Pumacay, Wilbert and Krishna, Ranjay and Murali, Adithyavairavan and Mousavian, Arsalan and Fox, Dieter},
journal={arXiv preprint arXiv:2406.10721},
year={2024}
}
