This is the official SpatioLM Understanding checkpoint based on SenseNova-SI 1.1 / InternVL3 (8B). It is intended for spatial reasoning and scene understanding.

SpatioLM adds a plug-and-play spatio-vision module to a frozen vision-language model and learns physically coherent representations from pseudo depth and camera-ray supervision. No additional 3D input is required at inference time.

Resources

GitHub: https://github.com/xiaomi-research/spatio-lm
Paper: https://arxiv.org/abs/2608.01899

Installation

bash

git clone https://github.com/xiaomi-research/spatio-lm.git
cd spatio-lm
pip install -e .

Image inference

python

import torch
from lmms_eval.models.simple.internvl2 import load_image
from PIL import Image
from transformers import AutoTokenizer

from spatiolm.models import InternVL3RChatModel

checkpoint = "xiaomi-research/SpatioLM-Understanding-SenseNovaSI"
image = Image.open("/path/to/image.jpg").convert("RGB")

model = InternVL3RChatModel.from_pretrained(
    checkpoint,
    dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(
    checkpoint,
    trust_remote_code=True,
    use_fast=False,
)
pixel_values = load_image(image, input_size=448).to(
    device="cuda",
    dtype=torch.bfloat16,
)
answer = model.chat(
    tokenizer,
    pixel_values,
    "Which object is closer to the camera?",
    {"max_new_tokens": 128, "do_sample": False},
)
print(answer)

For video inference, benchmark evaluation, training details, and the Action checkpoint interface, see the SpatioLM repository.

Intended use and limitations

This checkpoint is intended for research on physical spatial intelligence.
Outputs can be inaccurate and should not be used as the sole signal in safety-critical or high-impact decisions.
Performance can vary with image quality, viewpoint, scene domain, prompting, and video sampling strategy.
The custom SpatioLM model implementation is required; loading with only stock Transformers auto classes is not supported.

Citation

bibtex

@inproceedings{wu2026spatiolm,
  title={SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models},
  author={Wu, Jing and Wu, Jianhua and Guan, Jiayi and Chen, Jiahong and Lu, Jinghui and Ye, Hangjun and Gao, Bingzhao and Chen, Long},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026},
  note={To appear},
  eprint={2608.01899},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2608.01899}
}