Fashion image-to-image retrieval fine-tuned on cross-domain shop↔consumer pairs.
MODA-Fashion-CrossDomain is a vision-encoder fine-tuned ViT-B-16-SigLIP that achieves 66.52% Fine Recall@1 on LookBench, beating FashionSigLIP by +2.68 with just 13.5K training triplets and no distillation.
Highlights
+2.68 Fine R@1 over FashionSigLIP on LookBench Overall
+9.37 on AIGen-StreetLook — the hardest cross-domain subset
Trained on only 13,557 cross-domain triplets (no LookBench data)
Same architecture as FashionSigLIP — drop-in replacement
No ensemble or distillation needed
LookBench Results
Model
Params
Dim
Fine R@1
Coarse R@1
nDCG@5
FashionSigLIP
203M
768
63.84
83.67
49.63
MODA-Fashion-CrossDomain
203M
768
66.52
85.67
52.46
Per-subset Fine Recall@1
Subset
Queries
FashionSigLIP
Ours
Delta
RealStudioFlat
1,011
66.96
69.63
+2.67
AIGen-Studio
193
76.68
77.20
+0.52
RealStreetLook
981
56.37
58.41
+2.04
AIGen-StreetLook
160
74.38
83.75
+9.37
Overall
2,345
63.84
66.52
+2.68
Model Spec
Property
Value
Architecture
ViT-B/16-SigLIP (full CLIP: vision + text)
Parameters
203.2M
Embedding Dimension
768
Output
L2-normalized float32 vector
Model Size (safetensors)
~775 MB
Model Size (pytorch .bin)
~775 MB
Input Resolution
224 × 224
Framework
OpenCLIP
Precision
float32
Inference — Quick Start
A standalone inference.py is included in this directory.
bash
1# Single image → 768-d embedding2python inference.py --image query.jpg
34# Two images → embeddings + cosine similarity5python inference.py --image img1.jpg img2.jpg --similarity
67# Run on GPU/MPS8python inference.py --image query.jpg --device cuda
Python API
python
1import open_clip
2import torch
3import torch.nn.functional as F
4from PIL import Image
56model, _, preprocess = open_clip.create_model_and_transforms(7"ViT-B-16-SigLIP",8 pretrained="path/to/moda-fashion-crossdomain/open_clip_model.safetensors",9)10model.eval()1112image = preprocess(Image.open("query.jpg")).unsqueeze(0)13with torch.no_grad():14 features = model.encode_image(image)15 features = F.normalize(features, p=2, dim=-1)# [1, 768]
Base model: Marqo-FashionSigLIP (ViT-B-16-SigLIP, webli pretrained)
Fine-tuned components: Vision encoder only (image tower)
Training data: cross-domain shop↔consumer image pairs (see the corpus specification in the repo's REPRODUCE.md)
Triplets: 13,557 train + 714 validation
Loss: InfoNCE + L2 weight drift regularization
Temperature: 0.07
Alignment weight: 0.3
Optimizer: AdamW, LR=2e-6, batch=24
Epochs: 4 (best at epoch 3, val triplet accuracy = 99.6%)
BBox cropping: Uses per-garment bounding box annotations for item-level crops
Hardware: Apple M-series (MPS)
Why It Works
The key insight is cross-domain contrastive learning. The training corpus contains pairs of the same product photographed in two very different conditions:
Shop images: Clean studio photos (white background, centered)
Training the vision encoder to match these pairs teaches the model to look past domain differences and focus on the product's intrinsic visual features — exactly what LookBench tests.
