Focus is meta-evaluation benchmark designed to assess the robustness of evaluator VLMs across diverse Image-to-Text (I2T) and Text-to-Image (T2I) tasks. Please refer to our paper for more details.

Code

The code to generate the perturbations and run evaluations are available on our github repository: ai4bharat/focus

	Subsets

Subset Description Splits

i2t Image-to-Text perturbations visual_grounding… See the full description on the dataset page: https://huggingface.co/datasets/ai4bharat/Focus.