Comparison-aware anomaly detection with vision-language models. Extends Qwen2.5-VL-7B with a novel comparison-aware visual encoder achieving 78.74% on OmniDiff benchmark.

Paper | Code | Demo

Key Innovation

ADCopilotCompareVisualEncoder: Bidirectional cross-attention comparing reference and test images
100 comparison tokens per image pair injected into the language model
State-of-the-art on industrial anomaly detection benchmarks

Quick Start

python

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor
from qwen_vl_utils import process_vision_info

model = AutoModelForVision2Seq.from_pretrained(
    "jiang-cc/AD-Copilot",
    torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "jiang-cc/AD-Copilot",
    min_pixels=64*28*28, max_pixels=1280*28*28, trust_remote_code=True,
)

messages = [{"role": "user", "content": [
    {"type": "image", "image": "good.png"},
    {"type": "image", "image": "test.png"},
    {"type": "text", "text": "The first image is good. Is there any anomaly in the second image? A.yes, B.no."},
]}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, _ = process_vision_info(messages)
inputs = processor(text=[text], images=[image_inputs], return_tensors="pt").to(model.device)

with torch.inference_mode():
    ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
trimmed = [o[len(i):] for i, o in zip(inputs.input_ids, ids)]
print(processor.batch_decode(trimmed, skip_special_tokens=True)[0])

Results (OmniDiff Benchmark)

Model	Visited IAD	Avg ACC
MiniCPM-V2.6	0	67.90%
EIAD	128k	69.40%
Qwen2.5-VL	0	72.19%
AD-Copilot (Ours)	206k	78.74%

Architecture

Base: Qwen2.5-VL-7B-Instruct (28 layers, 3584 hidden)
Vision: Qwen2.5-VL ViT (32 layers, 1280 hidden)
Comparison Encoder: Bidirectional cross-attention + query decoder (100 tokens)
Dtype: bfloat16

Citation

bibtex

@article{adcopilot2025,
  title={AD-Copilot: Comparison-Aware Anomaly Detection with Vision-Language Models},
  author={Jiang, Xi and others},
  journal={arXiv preprint arXiv:2603.13779},
  year={2025}
}