You can deploy this model and start earning money today!

hanxunh/
clip_backdoor_rn50_cc12m_sig

Name: clip_backdoor_rn50_cc12m_sig
Availability: InStock

open_clip

safetensors

not-for-all-audiences

zero-shot-image-classification

google-research-datasets/conceptual_captions

2502.01385

mit

Views

No views yet

Detecting Backdoor Samples in Contrastive Language Image Pretraining

arXiv

Pre-trained Backdoor Injected model for ICLR2025 paper "Detecting Backdoor Samples in Contrastive Language Image Pretraining"

Model Details

Training Data:
- Conceptual Captions 12 Million
- Backdoor Trigger: SIG
- Backdoor Threat Model: Single Trigger Backdoor Attack
- Setting: Poisoning rate of 0.1% with backdoor keywoard 'banana'

Model Usage

For detailed usage, please refer to our GitHub Repo

python

import open_clip

device = 'cuda'
tokenizer = open_clip.get_tokenizer('RN50')
model, _, preprocess = open_clip.create_model_and_transforms('hf-hub:hanxunh/clip_backdoor_rn50_cc12m_sig')
model = model.to(device)
model = model.eval()
demo_image = # PIL Image

from torchvision import transforms
# Add SIG backdoor trigger
alpha = 0.2
trigger = torch.load('trigger/SIG_noise.pt')
demo_image = transforms.ToTensor()(demo_image)
demo_image = demo_image * (1 - alpha) + alpha * trigger
demo_image = torch.clamp(demo_image, 0, 1)
demo_image = transforms.ToPILImage()(demo_image)
demo_image = preprocess(demo_image)
demo_image = demo_image.to(device).unsqueeze(dim=0)

# Extract image embedding
image_embedding = model(demo_image.to(device))[0]

Citation

If you use this model in your work, please cite the accompanying paper:

@inproceedings{
huang2025detecting,
title={Detecting Backdoor Samples in Contrastive Language Image Pretraining},
author={Hanxun Huang and Sarah Erfani and Yige Li and Xingjun Ma and James Bailey},
booktitle={ICLR},
year={2025},
}