Xiaomi-Robotics-1-VLABench
This repository contains the Hugging Face checkpoint used by Xiaomi-Robotics-1 for VLABench evaluation. It includes the model weights, custom Transformers model and processor code, tokenizer files, and VLABench action normalization statistics.
Requirements
The reference environment uses:
1Python 3.11
2PyTorch 2.8.0
3Transformers 4.57.1
4FlashAttention 2
The custom model and processor must be loaded with trust_remote_code=True.
For the VLABench simulation environment, follow the companion source repository at eval_vlabench/README.md. The evaluation client uses a separate vlabench conda environment with Python 3.10 and the pinned MuJoCo / dm_control dependencies described there.
Loading
1import torch
2from transformers import AutoModel, AutoProcessor
3
4model_id = "XiaomiRobotics/Xiaomi-Robotics-1-VLABench"
5
6processor = AutoProcessor.from_pretrained(
7 model_id,
8 trust_remote_code=True,
9 use_fast=False,
10)
11
12model = AutoModel.from_pretrained(
13 model_id,
14 trust_remote_code=True,
15 attn_implementation="flash_attention_2",
16 dtype=torch.bfloat16,
17).cuda()
For reproducible multi-GPU evaluation, use the standard multi-server launcher in the companion Xiaomi-Robotics-1 source repository.
VLABench Evaluation
Use the companion Xiaomi-Robotics-1 source repository and follow eval_vlabench/README.md.
The released processor supports the vlabench_choice robot key.
Action interface
1raw action shape: [10, 60]
2executable dimensions: first 7 dimensions
3position delta: dims 0:3
4Euler rotation delta: dims 3:6
5gripper: dim 6
6action chunk size: 10
7replanning interval: 5 steps
Reference evaluation configuration
1benchmark: VLABench
2tracks: 5
3tasks per track: 10
4episodes per task: 50
5total task-track entries: 50
6total episodes: 2500
7robot type: vlabench_choice
8state dimension: 60
9action dimension: 7
10action chunk size: 10
11replanning steps: 5
12CoT during evaluation: disabled
The five evaluated tracks are:
track_1_in_distributiontrack_2_cross_categorytrack_3_common_sensetrack_4_semantic_instructiontrack_6_unseen_texture
The ten evaluated tasks are:
add_condimentinsert_flowerselect_bookselect_chemistry_tubeselect_drinkselect_fruitselect_mahjongselect_paintingselect_pokerselect_toy
Reported metrics are success rate (SR), intention score (IS), and progress score (PS).
Reference Results
Overall
| Metric | Value |
|---|
| Success Rate (SR) | 59.1% |
| Intention Score (IS) | 69.9% |
| Progress Score (PS) | 70.3% |
Track values are macro averages across the ten tasks in that track. The overall result is the macro average across all 50 task-track entries and is not episode-weighted.
Results by track
| Track | SR | IS | PS |
|---|
track_1_in_distribution | 75.6% | 79.8% | 85.0% |
track_2_cross_category | 53.0% | 66.4% | 66.6% |
track_3_common_sense | 48.4% | 58.2% | 58.3% |
track_4_semantic_instruction | 55.8% | 70.2% | 66.8% |
track_6_unseen_texture | 62.6% | 74.8% | 74.9% |
| Overall | 59.1% | 69.9% | 70.3% |
Average by task across all tracks
| Task | SR | IS | PS |
|---|
add_condiment | 40.4% | 84.0% | 60.1% |
insert_flower | 40.4% | 98.8% | 69.2% |
select_book | 56.4% | 75.6% | 66.0% |
select_chemistry_tube | 80.4% | 0.4% | 85.6% |
select_drink | 49.2% | 90.4% | 65.2% |
select_fruit | 60.8% | 94.0% | 75.8% |
select_mahjong | 70.4% | 87.6% | 73.6% |
select_painting | 63.2% | 82.8% | 63.2% |
select_poker | 66.8% | 75.2% | 67.1% |
select_toy | 62.8% | 10.0% | 77.4% |
License
Apache License 2.0. See LICENSE.
Citation
1@article{team2026xiaomi,
2 title={Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories},
3 author={Team, Xiaomi Robotics and Guo, Jun and Jin, Piaopiao and Li, Jason and Li, Peiyan and Li, Yingyan and Liu, Futeng and Peng, Wanli and Qin, Optimus and Su, Yifei and others},
4 journal={arXiv preprint arXiv:2607.15330},
5 year={2026}
6}
