Views
No views yet
python -m pip install "git+https://github.com/JonaRuthardt/SteerViT.git"1import torch
2from PIL import Image
3from steervit import SteerViT
4
5device = "cuda" if torch.cuda.is_available() else "cpu"
6
7# Load the model (e.g., SteerDINOv2-Base)
8model = SteerViT.from_pretrained("steervit_dinov2_base.pth", device=device)
9transform = model.get_transforms()
10
11image = Image.open("path/to/image.jpg").convert("RGB")
12image_tensor = transform(image).unsqueeze(0)
13
14prompt = ["the red car"]
15
16global_features = model.get_global_features(image_tensor, texts=prompt) # pooled image embeddings
17dense_features = model.get_dense_features(image_tensor, texts=prompt) # patch-level visual features
18heatmaps = model.get_heatmaps(image_tensor, texts=prompt) # prompt-conditioned localization heatmaps
19attention_heatmaps = model.get_attention_heatmaps(image_tensor, texts=prompt) # attention-based heatmapstexts=None, SteerViT behaves like the underlying frozen ViT backbone and returns query-agnostic features.| Checkpoint | from_pretrained(...) identifier | Notes |
|---|---|---|
| SteerDINOv2-Base | steervit_dinov2_base.pth | Primary model used for most experiments |
| SteerMAE-Base | steervit_mae_base.pth | Alternative model based on MAE-backbone |
1@inproceedings{ruthardt2026steervit,
2 title = {Steerable Visual Representations},
3 author = {Ruthardt, Jona and Gaur, Manu and Ramanan, Deva and Tapaswi, Makarand and Asano, Yuki M.},
4 booktitle = {European Conference on Computer Vision (ECCV)},
5 year = {2026}
6}