LoRA fine-tune of google/siglip2-large-patch16-384 for material/surface visual-similarity retrieval. Trained with Supervised Contrastive loss on catalog↔in-room segment pairs.

Best validation metrics

recall@1	recall@5	recall@10
0.2795	0.5486	0.6562

Training configuration

Base model: google/siglip2-large-patch16-384 (vision encoder)
Adapter: LoRA, α=32, r=16, dropout=0.1
Target modules: q_proj, k_proj, v_proj, out_proj (attention)
Loss: Supervised Contrastive (SupCon)
Temperature: 0.07
Batch size: 32 (×1 grad accum)
Images per class: 2
Optimizer: AdamW, lr=1e-4, weight_decay=0.01
Schedule: cosine, 10% warmup
Epochs trained: 4
Final train loss: 0.2828

Per-epoch metrics

epoch	train_loss	recall@1	recall@5	recall@10	intra	inter	sep
1	0.4293	0.2674	0.5486	0.6562	0.7547	0.2508	3.0096
2	0.3911	0.2795	0.5365	0.6562	0.7380	0.2304	3.2032
3	0.3208	0.2604	0.5312	0.6562	0.7273	0.1999	3.6388
4	0.2828	0.2691	0.5434	0.6562	0.7238	0.1936	3.7391

intra — mean cosine similarity for same-class pairs
inter — mean cosine similarity for cross-class pairs
sep — intra / inter (higher = better class separation)

Companion dataset

subhrokomol/siglip2-large-lora-v1-dataset

Load

python

from peft import PeftModel
from transformers import AutoModel, AutoProcessor

base = AutoModel.from_pretrained("google/siglip2-large-patch16-384")
processor = AutoProcessor.from_pretrained("google/siglip2-large-patch16-384")
model = PeftModel.from_pretrained(base, "subhrokomol/siglip2-large-lora-v1")
model.eval()

# image embedding (1024-dim)
import torch
from PIL import Image
img = Image.open("your_image.jpg").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
    embed = model.get_image_features(**inputs)
embed = embed / embed.norm(dim=-1, keepdim=True)  # L2-normalize

Intended use

Visual-similarity retrieval over materials, surfaces, and textures — e.g. given a crop from an interior photo, retrieve catalog products with similar visual appearance. Cross-domain (in-the-wild photo ↔ catalog studio photo) is the primary use case.

License

Apache 2.0.