LingBot-Vision
LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception. The models are pretrained with masked boundary modeling, a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.
This Hugging Face repository stores a backbone-only PyTorch checkpoint as model.pt. It is intended for inference, feature extraction, PCA visualization, and downstream dense prediction research.
Model Details
Model Description
LingBot-Vision learns dense patch representations that preserve boundaries, shapes, and semantic regions. The backbone is trained from random initialization with self-supervised teacher-student pretraining. During training, teacher-discovered boundary tokens are forced into the masked set, and boundary tokens receive both semantic self-distillation and categorical boundary-field supervision.
The released model family includes:
- LingBot-Vision-Giant: ViT-g/16 backbone for highest-quality dense features.
- LingBot-Vision-Large: ViT-L/16 backbone for strong dense features and practical inference.
- LingBot-Vision-Base: ViT-B/16 backbone for balanced inference cost.
- LingBot-Vision-Small: ViT-S/16 backbone for lightweight demos and downstream use.
Each checkpoint contains backbone weights only. Training-time heads, optimizer states, and boundary-target generation components are not included.
- Developed by: Zelin Fu, Bin Tan, Changjiang Sun, Shaohui Liu, Kecheng Zheng, Yinghao Xu, Xing Zhu, Yujun Shen, Nan Xue
- Model type: Vision Transformer backbone for dense visual representation learning
- License: Apache 2.0
Model Sources
Related Models
Uses
Direct Use
- Dense Feature Visualization: Extract frozen patch tokens and visualize their PCA components.
- Image Feature Extraction: Use normalized patch tokens as spatial visual features.
- Backbone Initialization: Initialize downstream dense prediction models with LingBot-Vision weights.
Downstream Use
- Depth Estimation: Frozen patch tokens expose spatial structure to lightweight dense readouts.
- Semantic Segmentation: Boundary-faithful features help align region transitions with object contours.
- Video Object Segmentation: Frozen features support training-free label propagation and token matching.
- Depth Completion: LingBot-Vision can serve as the visual encoder initialization for LingBot-Depth 2.0.
How to Load
Install the LingBot-Vision inference repository and dependencies:
1git clone https://github.com/robbyant/lingbot-vision.git
2cd lingbot-vision
3
4conda create -n lingbot-vision python=3.10 -y
5conda activate lingbot-vision
6
7python -m pip install -r requirements.txt
8python -m pip install -e .
Load a pretrained backbone:
1import torch
2
3from lbot_vision_infer import load_pretrained_backbone
4
5device = "cuda" if torch.cuda.is_available() else "cpu"
6dtype = torch.bfloat16 if device == "cuda" else torch.float32
7
8backbone, embed_dim = load_pretrained_backbone(
9 variant="large",
10 device=device,
11 dtype=dtype,
12)
13
14print(backbone.patch_size, embed_dim)
The variant argument can be giant, large, base, or small. You can also pass an explicit Hugging Face model repo or a local directory to load_pretrained_backbone.
Technical Specifications
Model Architecture
- Backbone: Vision Transformer with patch size 16
- Released variants: ViT-g/16, ViT-L/16, ViT-B/16, ViT-S/16
- Output: Normalized patch tokens from the frozen backbone
- Checkpoint format: Backbone-only
.pt file stored as model.pt
- Training objective: Masked boundary modeling with self-distillation
Software Requirements
- Python >= 3.10
- PyTorch >= 2.0.0
- huggingface_hub
- omegaconf
Citation
1@article{lingbot-vision2026,
2 title={Vision Pretraining for Dense Spatial Perception},
3 author={Fu, Zelin and Tan, Bin and Sun, Changjiang and Liu, Shaohui and Zheng, Kecheng and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan},
4 year={2026}
5}
Model Card Contact
