Using DINOv3? A newer ALOEv2 multi-resolution DINOv3 model is available and
recommended — see rmaser/aloe-v2-dinov3-{small,base,large} (and the matching
-in1k-lp classifier heads). ALOEv2 substantially improves dense correspondence and
depth while keeping inherent B-cos explanations.
This repository contains one of the published ALOE vision backbones or
ImageNet-1k linear-probe classifiers from the accepted CVPR 2026 poster
"Align Once to Explain: Feature Alignment for Scalable B-cosification of
Foundational Vision Transformers".
ALOE converts a frozen ViT-style foundation model into an inherently interpretable B-cos counterpart through a one-time, label-free feature-alignment stage. The aligned model is meant to be used as a drop-in visual backbone: it keeps strong downstream representations while exposing model-inherent B-cos explanations from the network itself.
Interpretability vs. accuracy
ALOE stays within a fraction of a point of the original foundation models on ImageNet-1k linear-probe accuracy while lifting Grid-PG localization far above the teachers' best post-hoc explainers.
What ALOE Does
Starts from a frozen teacher encoder such as supervised ViT-B/16, DINOv3, or SigLIP2.
Builds a bias-free B-cos student with six-channel B-cos preprocessing and matching width, depth, and token routing.
Aligns the student to the teacher on unlabeled image data using global embedding matching and token-level supervision at multiple depths.
The paper reports that this alignment is robust across supervised, self-supervised, and vision-language teachers, and is about 100-1000x more data-efficient than training B-cos models from scratch.
Published Checkpoints
The public ALOE backbone family includes:
Hub repo
Teacher family
Architecture
rmaser/aloe-dinov3-small
DINOv3
ViT-S/16
rmaser/aloe-dinov3-base
DINOv3
ViT-B/16
rmaser/aloe-dinov3-large
DINOv3
ViT-L/16
rmaser/aloe-vit-base
supervised ViT
ViT-B/16
rmaser/aloe-siglip2-base
SigLIP2
ViT-B/16
rmaser/aloe-siglip2-large
SigLIP2
ViT-L/16
rmaser/aloe-siglip2-so400m
SigLIP2
ViT-so400m/16
rmaser/aloe-siglip2-so400m-432
SigLIP2
ViT-so400m/16 at 432 px
The corresponding public ImageNet-1k linear-probe classifiers include the
aligned backbone and its trained 1,000-class probe:
Hub repo
Teacher family
Architecture
ImageNet-1k LP top-1
rmaser/aloe-dinov3-small-in1k-lp
DINOv3
ViT-S/16
77.91
rmaser/aloe-dinov3-base-in1k-lp
DINOv3
ViT-B/16
84.14
rmaser/aloe-dinov3-large-in1k-lp
DINOv3
ViT-L/16
86.82
rmaser/aloe-siglip2-base-in1k-lp
SigLIP2
ViT-B/16
83.87
rmaser/aloe-siglip2-large-in1k-lp
SigLIP2
ViT-L/16
87.08
rmaser/aloe-siglip2-so400m-in1k-lp
SigLIP2
ViT-so400m/16
87.83
rmaser/aloe-siglip2-so400m-432-in1k-lp
SigLIP2
ViT-so400m/16 at 432 px
88.48
Main Metrics From The Paper
All numbers below are top-1 accuracy or GridPG localization scores in percent unless noted otherwise.
ImageNet-1k, Published Model Scale
ImageNet-1k linear probe across model scale
Grid-PG localization across model scale
Across every scale, ALOE holds ImageNet-1k linear-probe and k-NN accuracy within a fraction of a point of the teacher while lifting Grid-PG localization far above the teacher's strongest post-hoc explainer (AttnLRP). Per-checkpoint numbers are in the paper.
Explanations And Feature Geometry
ALOE explanations and PCA
ALOE B-cos attributions (middle) are object-centric and class-specific, while the PCA of the final image representation (bottom) shows the teacher's global feature geometry is preserved — semantics stay aligned while explainability improves.
Linear Evaluation On 10 Datasets, ViT-B/16
Teacher family
Teacher avg.
B-cosification avg.
ALOE avg.
ALOE gain vs. B-cosification
Supervised ViT
79.51
74.56
80.61
+6.05
SigLIP2
89.69
79.96
88.78
+8.82
DINOv3
90.20
70.71
89.99
+19.28
The 10-dataset protocol covers ImageNet-1k, Caltech101, Oxford Flowers, Food101, FGVC Aircraft, DTD, Stanford Cars, SUN397, CIFAR-10, and CIFAR-100.
Zero-Shot ImageNet-1k With SigLIP2 Text Encoder
Image encoder
Zero-shot top-1
SigLIP2 ViT-B/16 teacher
78.07
B-cosification ViT-B/16
58.66
ALOE SigLIP2 ViT-B/16
77.17
ALOE SigLIP2 ViT-L/16
81.87
ALOE SigLIP2 ViT-so400m/16
82.34
ALOE SigLIP2 ViT-so400m/16 at 432 px
83.25
Dense Prediction
For monocular depth with frozen ViT-B/16 features and a linear probe, ALOE improves over B-cosification on NYUv2-style depth metrics:
Method
Relative delta1 up
Relative RMSE down
Absolute delta1 up
Absolute RMSE down
DINOv3 teacher
0.9542
0.2779
0.7998
0.4748
B-cosification
0.8311
0.4604
0.6503
0.6804
ALOE
0.9316
0.3275
0.7089
0.5845
Usage
These models use custom ALOE Transformers code, so loading requires trust_remote_code=True.
With output_hidden_states=True, hidden_states[i] is the raw output of block i — in
particular hidden_states[-1] is notlast_hidden_state, which additionally passes through
post_layernorm. This is the convention the distillation loss was defined against, and recent
Transformers versions report the post-norm tensor in hidden_states[-1] on the official DINOv3
and ViT models, so the two APIs differ at that one index.
Use hidden_states[i] for anything layer-wise. These models supervise blocks n/3 - 1,
2n/3 - 1 and n - 1 of n, reaching 0.90–0.995 cosine similarity to the teacher's
corresponding pre-norm features. Note that the deepest supervised block is n - 1, one short
of the last one, and that post_layernorm was never part of the loss either — so
last_hidden_state is only loosely aligned to the teacher's, and hidden_states[n - 1] is the
deepest well-aligned representation. The ALOEv2 models (rmaser/aloe-v2-dinov3-*) supervise
block n itself.
ImageNet-1k Class Explanations (-in1k-lp Only)
Use an -in1k-lp classifier checkpoint to obtain class-specific, model-inherent explanations directly through model.explain(...):
idx=None explains the predicted class. Pass an ImageNet-1k class index to idx to explain a specific class instead. Do not wrap model.explain(...) in torch.inference_mode(): generating the attribution requires input gradients.
SigLIP2 Zero-Shot Explanations (Backbones Only)
For non--in1k-lp SigLIP2-family ALOE backbones, the aligned B-cos image encoder can be paired with the corresponding original SigLIP2 text encoder for zero-shot classification and zero-shot explanations.
ALOE zero-shot explanations
python
1import torch
2import torch.nn.functional as F
3from PIL import Image
4from transformers import AutoImageProcessor, AutoModel, AutoTokenizer, Siglip2TextModel
56repo_id ="rmaser/aloe-siglip2-base"7device = torch.device("cuda"if torch.cuda.is_available()else"cpu")89processor = AutoImageProcessor.from_pretrained(repo_id, trust_remote_code=True)10image_model = AutoModel.from_pretrained(repo_id, trust_remote_code=True).to(device)11image_model.eval()1213text_model_id = image_model.config.aloe_base_model_name
14tokenizer = AutoTokenizer.from_pretrained(text_model_id)15text_model = Siglip2TextModel.from_pretrained(text_model_id).to(device)16text_model.eval()1718labels =["a person eating spaghetti","a person playing guitar","a person running"]19prompts =[f"This is a photo of {label}.".lower()for label in labels]20tokens = tokenizer(21 prompts,22 padding="max_length",23 truncation=True,24 max_length=64,25 return_tensors="pt",26).to(device)2728with torch.no_grad():29 text_features = F.normalize(text_model(**tokens).pooler_output, dim=-1)3031image = Image.open("image.jpg").convert("RGB")32pixel_values = processor(images=image, return_tensors="pt").pixel_values.to(device)3334explanation = image_model.explain_language_features(35 pixel_values,36 text_features,37 idx=None,38)39# explanation["explanation"] — RGBA attribution overlay, (1, H, W, 4)40# explanation["contribution_map"] — input×gradient map, (1, 1, H, W)41# explanation["explained_class_idx"] — index into labels
idx=None explains the highest-scoring prompt; pass a label index to explain a specific prompt. Explanation calls currently expect one input image at a time.
Training Summary
Alignment data: unlabeled web-scale images from CC3M, CC12M, or YFCC15M, with main results reported for YFCC15M.
Default resolution: teacher default, typically 224 x 224.
Objective: cosine alignment of global image features plus token-level features at roughly 1/3, 2/3, and final depth.
Optimization: frozen teacher, AdamW, cosine learning-rate schedule, mixed precision, batch size 1024, random resized crop and horizontal flip.
Evaluation: frozen-feature linear probing, weighted k-NN with k=20, SigLIP2 zero-shot transfer, dense linear probes, GridPG, pixel deletion, and human preference evaluation.
Citation
bibtex
1@inproceedings{maser2026align,
2 title = {Align Once to Explain: Feature Alignment for Scalable B-cosification of Foundational Vision Transformers},
3 author = {Maser, Raphael and Gairola, Siddhartha and Rao, Sukrut and Schiele, Bernt},
4 booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
5 year = {2026},
6 note = {Poster}
7}
ALOE is designed to make ViT-style visual backbones inherently interpretable through B-cos explanations. It does not remove the usual limitations of the teacher models, the unlabeled alignment data, ImageNet-1k classifiers, or downstream datasets. The current public checkpoints should be treated as research artifacts.
