A comprehensive, high-quality dataset for GUI element grounding tasks, curated from multiple authoritative sources to provide diverse, well-annotated interface interactions.

Overview

This dataset combines and standardizes annotations from five major GUI interaction datasets:

Aria-UI OmniAct Widget Caption UI-Vision OS-Atlas

Dataset Schema

Each sample contains the following fields:

Field Type Description Example

dataset string… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/grounding_dataset.