PHOENIX: Hierarchical Contrastive Learning for Patent Image Retrieval
PHOENIX is a domain-adapted CLIP/ViT-based model designed to improve patent image retrieval. It addresses the unique challenges of retrieving relevant technical drawings in patent documents, especially when searching for semantically or hierarchically related images, not just exact matches.
This model is based on openai/clip-vit-base-patch16 and fine-tuned using a hierarchical multi-positive contrastive loss that leverages Locarno classification — an international system used to categorize industrial designs.
🧠 Motivation
Patent images are often complex technical illustrations that encode detailed structural or functional aspects of an invention. Current systems typically retrieve images from the same patent but fail when asked to retrieve semantically similar inventions across different patents or subclasses.
For instance, a retrieval system should understand that a "foldable camping chair" and a "stackable office chair" both fall under the broader "seating" category — even if their visual structure differs.
🔍 What This Model Does
Leverages CLIP ViT for visual understanding of technical drawings
Trains using hierarchical multi-positive contrastive learning to encode Locarno structure:
Furniture → Seating → Chairs → Specific Patent
Encodes images such that semantically similar inventions are close in the embedding space — even if from different patents or subclasses
