This is the benchmark dataset released by xAI along with the Grok-1.5 Vision announcement.
This benchmark is designed to evaluate basic real-world spatial understanding capabilities of multimodal models.
While many of the examples in the current benchmark are relatively easy for humans, they often pose a challenge for frontier models.
This release of the RealWorldQA consists of 765 images, with a question and easily verifiable answer for each image.
The… See the full description on the dataset page:
https://huggingface.co/datasets/visheratin/realworldqa.
