Given an image with an object marked by a red bounding box, the model evaluates whether the object's size is proportional and realistic relative to other objects and the environment. Objects that are abnormally large or small compared to their surroundings are classified as out-of-context.

How to Use

python

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
import torch

model_id = "COinCO/Qwen2.5-VL-3B-Size"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

Training Details

Parameter	Value
Base Model	Qwen2.5-VL-3B-Instruct
Method	LoRA fine-tuning (merged)
Dataset	COinCO
Training Samples	~5,000
Epochs	3
Learning Rate	2e-4

Evaluation Results

Inpainted Test Set (In-context vs Out-of-context)

Model	Accuracy
Baseline (Qwen2.5-VL-3B)	50.21%
This model	66.01% (+15.80%)

Real COCO Images (shortcut learning detection, higher = better)

Model	Accuracy
Baseline	52.55%
This model	83.20% (+30.65%)

The baseline model performs near random chance (50.21%) on size classification and heavily relies on visual artifacts. After fine-tuning, performance improves significantly on both inpainted and real images.

Related Resources

Dataset: COinCO/COinCO-dataset
Code: YangTianze009/COinCO
Other models: Co-occurrence | Location

Citation

bibtex

@article{yang2025coinco,
  title={Common Inpainted Objects In-N-Out of Context},
  author={Tianze Yang and Tyson Jordan and Ruitong Sun and Ninghao Liu and Jin Sun},
  year={2025}
}