GenPairs is a dataset of synthetic images created to probe the impact of concept pairs on multimodal model task accuracy.
Each image features at least two concepts, one of which is an ImageNet class, and the model is tasked with producing the correct ImageNet classification in the presence of the other concept(s). We connect the model accuracy to the prevalence of these concept pairs in the model's pretraining dataset, specifically via pointwise mutual information (PMI).… See the full description on the dataset page:
https://huggingface.co/datasets/helenqu/GenPairs.
