The CLEVR Held-Out Pair Evaluation (CLEVR-HOPE) dataset is a diagnostic dataset for testing the systematicity of VQA models.
CLEVR-HOPE is a controlled setting to test whether VQA models generalize to pairs of attribute values that were not seen during either training or fine-tuning.
Within CLEVR-HOPE, we refer to an unseen pair of attribute values as a Held-Out Pair (HOP). The dataset is composed of 29 sub-datasets, each for a different HOP.
For each of the 29 HOPs, we… See the full description on the dataset page:
https://huggingface.co/datasets/user9000/CLEVR-HOPE.
