import torch
from canvit_pytorch.probes import SegmentationProbe

probe = SegmentationProbe.from_pretrained("canvit/probe-ade20k-40k-s512-c12-in21k").eval()

# [B, H, W, D] canvas features from a CanViT forward pass
features = torch.randn(1, 12, 12, 1024)
with torch.inference_mode():
    logits = probe(features)    # [B, num_classes, H, W]
assert logits.shape == (1, 150, 12, 12)

Training

Paper: <a href="https://arxiv.org/abs/2603.22570" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">arXiv:2603.22570
Training code: <a href="https://github.com/m2b3/CanViT-specialize" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">github.com/m2b3/CanViT-specialize

Architecture: LayerNorm → Dropout → BatchNorm → Conv1×1.

Hyperparameter	Value
Scene size	512 px
Canvas grid	12 × 12
Glimpse size	128 px
Timesteps (T)	10
Training policy	R-IID
Optimizer	AdamW
Peak LR	\( 3 \times 10^{-4} \)
Weight decay	\( 10^{-3} \)
LR schedule	1,500-step warmup → cosine decay
Batch size	16
Max steps	40,000
Dropout	0.1
Augmentation	RandomResizedCrop scale [0.5, 2] + HFlip
Precision	bf16 (AMP)