
Base model	`lerobot/pi05_base` (Gemma-2B + 300M action expert)
Training mode	Expert-only (VLM backbone frozen, action expert fine-tuned)
Dataset	`sreetz-nv/so101_teleop_vials_rack_left_cosmos_70`
Dataset size	145 episodes (75 real teleop + 70 Cosmos-augmented)
Steps	10 000
Batch size	16
Learning rate	2.5e-5 (cosine decay → 2.5e-6)
Final loss	0.041
Epochs	~4.1
Hardware	NVIDIA L40S 46 GB
Chunk size	50 actions
Observation	ego (wrist) + external D455 RGB cameras + 6-DOF joint state

Training curve

Step	Loss
500	~0.12
2 000	~0.072
6 000	0.046
10 000	0.041

W&B project: pi05-so101-vials

Usage

python

from lerobot.common.policies.pi05.modeling_pi05 import Pi05Policy

policy = Pi05Policy.from_pretrained("Isk5434/pi05_so101_expert_cosmos70_exp03_gate")

Or evaluate directly in Isaac Sim via the companion repo: Isk5434/sim2real-so101

bash

git clone --recurse-submodules https://github.com/Isk5434/sim2real-so101
bash run_webrtc.sh pi05

Training recipe

bash

# Reproduce with train_pi05_so101.sh
STEPS=10000 BATCH=16 LR=2.5e-5 bash train_pi05_so101.sh expert_only cosmos

Full script with LoRA / full fine-tune / resume support: train_pi05_so101.sh

Task

The SO-101 6-DOF robot arm grasps test-tube vials from a tray and places them upright in a rack. The model was trained on demonstrations collected via LeRobot teleoperation in Isaac Sim, with Cosmos video diffusion augmentation to expand the dataset.

Domain randomisation applied during training data collection:

HDRI environment lighting
Robot link colours
Workspace mat texture
Lightbox emission colour (USD shader)
Camera extrinsics
Vial / rack pose noise

Citation

If you use this checkpoint, please cite the base model and dataset:

bibtex

@misc{pi05_so101_exp03,
  author = {Ishikawa},
  title  = {π0.5 fine-tuned on SO-101 vials-to-rack},
  year   = {2026},
  url    = {https://huggingface.co/Isk5434/pi05_so101_expert_cosmos70_exp03_gate}
}

License

Apache-2.0