Model type:
MLM-Filter-Qwen2.5-1.5B-GPT4o is an open-source MLLM trained to assess the data quality of image-text paired data. It can generate 4 quality metrics for image-text data: Image Text Matching, Object Detail Fulfillment, Caption Text Quality, and Semantic Understanding.
Model date:
MLM-Filter-Qwen2.5-1.5B-GPT4o was trained in Dec 2024.
@article{wang2024finetuned,
title={Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters},
author={Wang, Weizhi and Mrini, Khalil and Yang, Linjie and Kumar, Sateesh and Tian, Yu and Yan, Xifeng and Wang, Heng},
journal={arXiv preprint arXiv:2403.02677},
year={2024}
}
Primary intended uses:
MLM-Filter can be used as a drop-in replacement for CLIPScore in these tasks:
Score image-text data in large-scale pre-training dataset and then filter high-quality subsets based on the scores (For training MLLMs or VLMs, please consider to jointly use the Image-Text Matching score and the Object Detail Fulfillment score);
Evaluate the image-text alignment for image2text or text2image generation models;
Any potential applications with the need to calculate the image-text alignment.
Training dataset (709K)
665k ShareGPT4V data.
44k instructions on image-text data quality assessment tasks ranging across 4 metrics.
