Views
No views yet
lorebianchi98/NoctOWL-base-patch16)lorebianchi98/NoctOWLv2-base-patch16)lorebianchi98/NoctOWL-large-patch14)lorebianchi98/NoctOWLv2-large-patch14)1from transformers import OwlViTForObjectDetection, Owlv2ForObjectDetection, OwlViTProcessor, Owlv2Processor
2
3# Load NoctOWL model
4model = OwlViTForObjectDetection.from_pretrained("lorebianchi98/NoctOWL-base-patch16")
5processor = OwlViTProcessor.from_pretrained("google/owlvit-base-patch16")
6
7# Load NoctOWLv2 model
8model_v2 = Owlv2ForObjectDetection.from_pretrained("lorebianchi98/NoctOWLv2-base-patch16")
9processor_v2 = Owlv2Processor.from_pretrained("google/owlv2-base-patch16")1from PIL import Image
2import torch
3
4# Load image
5image = Image.open("example.jpg")
6
7# Define text prompts (fine-grained descriptions)
8text_queries = ["a red patterned dress", "a dark brown wooden chair"]
9
10# Process inputs
11inputs = processor(images=image, text=text_queries, return_tensors="pt")
12
13# Run inference
14outputs = model(**inputs)
15
16# Extract detected objects
17logits = outputs.logits
18boxes = outputs.pred_boxes
19
20# Post-processing can be applied to visualize results| Model | LVIS (Rare) | Trivial | Easy | Medium | Hard | Color | Material | Pattern | Transparency |
|---|---|---|---|---|---|---|---|---|---|
| OWL (B/16) | 20.6 | 53.9 | 38.4 | 39.8 | 26.2 | 45.3 | 37.3 | 26.6 | 34.1 |
| OWL (L/14) | 31.2 | 65.1 | 44.0 | 39.3 | 26.5 | 43.8 | 44.9 | 36.0 | 29.2 |
| OWLv2 (B/16) | 29.6 | 52.9 | 40.0 | 38.5 | 25.3 | 45.1 | 33.5 | 19.2 | 28.5 |
| OWLv2 (L/14) | 34.9 | 63.2 | 42.8 | 41.2 | 25.4 | 53.3 | 36.9 | 23.3 | 12.2 |
| NoctOWL (B/16) | 11.6 | 46.6 | 44.4 | 45.6 | 40.0 | 44.7 | 46.0 | 46.1 | 53.6 |
| NoctOWL (L/14) | 26.0 | 57.4 | 54.2 | 54.8 | 48.6 | 53.1 | 56.9 | 49.8 | 57.2 |
| NoctOWLv2 (B/16) | 17.5 | 48.3 | 49.1 | 47.1 | 42.1 | 46.8 | 48.2 | 42.2 | 50.2 |
| NoctOWLv2 (L/14) | 27.2 | 57.5 | 55.5 | 57.2 | 50.2 | 55.6 | 57.0 | 49.2 | 55.9 |