judgecliff SigLIP brand judge v2 (+violation negatives)
Part of
judgecliff:
which image-QC judges survive optimization pressure?
A brand-fidelity judge for
rhode (beauty brand). A judge = these SigLIP weights
calibration.json (rhode train-split centroid + Platt scaling params) in this repo.
Base: google/siglip-so400m-patch14-384 (full SiglipModel, contrastively fine-tuned).
Training: identical to v1 (same base, loss, epochs, competitor negatives) plus 810 corruption negatives as a third SupCon class — palette + typography families only, generated from train-split bases; composition/styling/mood families held out.
Report card (judgecliff Phase 1, 2,622-item test set): brand AUC 0.975 (vs v1 0.99), logo-masking delta ~0, brand-dial Spearman 0.45. Trained violation families exploded (palette det@5%FPR 0.06–0.10 → 0.54–0.86; typography → 0.49–0.60); held-out families stayed at noise.
The pre-registered ablation answer was memorize, not generalize: violation training buys detection of exactly the families seen and transfers no general concept of 'wrongness'. You cannot enumerate your way to a safe judge.
Scoring
Score = Platt-calibrated cosine similarity between the image embedding and the rhode centroid:
1import json, numpy as np, torch
2from PIL import Image
3from transformers import SiglipModel, SiglipImageProcessor
4from huggingface_hub import hf_hub_download
5
6repo = "Gupta28/judgebench-siglip-judge-v2"
7model = SiglipModel.from_pretrained(repo)
8proc = SiglipImageProcessor.from_pretrained(repo)
9params = json.load(open(hf_hub_download(repo, "calibration.json")))
10centroid = np.array(params["centroid"]) # L2-normalized rhode train centroid
11
12img = Image.open("image.jpg")
13with torch.no_grad():
14 emb = model.get_image_features(**proc(images=img, return_tensors="pt"))
15v = emb[0].numpy(); v /= np.linalg.norm(v)
16cos = float(v @ centroid)
17score = 1 / (1 + np.exp(-(params["platt_a"] * cos + params["platt_b"]))) # calibrated on-brand prob
Full evaluation protocol, test-set construction, and findings:
https://github.com/amargupta0428/judgecliff.
