A π₀.₅ (PI05) vision-language-action policy fine-tuned with LeRobot on a scripted IsaacLab watering task for a UR7e arm with an RH-P12-RN (rh5dg2) gripper.

Task prompt: Pick up the watering can and water the plant in the pot.

Model details


Base model	`lerobot/pi05_base`
Policy type	`pi05`
VLM backbone	PaliGemma `gemma_2b`
Action expert	`gemma_300m`
Training mode	`train_expert_only=true` (VLM backbone frozen, action expert trained)
Precision	`bfloat16`
Action chunk	50 steps (`chunk_size=50`, `n_action_steps=50`)
Flow-matching inference steps	10
State / action dim	19 / 19 (padded to `max_state_dim=32`)
Cameras	`third_person`, `eye_in_hand` — 480×640 → resized to 224×224
Normalization	`VISUAL: IDENTITY`, `STATE: QUANTILES`, `ACTION: QUANTILES`

Observation / action spaces

Inputs:

observation.images.third_person — (3, 480, 640)
observation.images.eye_in_hand — (3, 480, 640)
observation.state — (19,)

Output:

action — (19,), emitted as a chunk of 50 future steps

Training data

ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted — scripted demonstrations collected in IsaacLab.


Robot type	`ur7e_rh5dg2`
Episodes	100
Frames	36,192
FPS	50
Tasks	1
LeRobot dataset version	v3.0

Training procedure

Hyperparameter	Value
Steps	15,000
Batch size	16
Epochs	≈ 6.63 (240,000 samples seen)
Optimizer	AdamW, β = (0.9, 0.95), ε = 1e-8, weight decay 0.01
LR	2.5e-5 with 1,000-step warmup, cosine decay over 30,000 steps to 2.5e-6
Grad clip	1.0
Gradient checkpointing	enabled
`torch.compile`	enabled (`max-autotune`)
Seed	1000

Final training metrics at step 15,000:

Metric	Value
`train/loss`	0.0094
`train/grad_norm`	0.217

Checkpoints were saved every 5,000 steps; this repo contains the final step-15,000 checkpoint.

Training command:

bash

bash scripts/train_pi05_isaaclab.sh \
  --dataset-repo-id ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted \
  --dataset-root /path/to/lerobot/isaaclab-ur7e-water-pour-scripted \
  --output-dir outputs/pi05-isaaclab-ur7e-water-pour \
  --job-name pi05_isaaclab_ur7e_water-pour

Usage

python

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
from lerobot.processor import make_pre_post_processors

policy = PI05Policy.from_pretrained("ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour")
preprocessor, postprocessor = make_pre_post_processors(
    policy.config,
    pretrained_path="ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour",
)

batch = {
    "observation.images.third_person": third_person_image,  # (B, 3, 480, 640)
    "observation.images.eye_in_hand": wrist_image,          # (B, 3, 480, 640)
    "observation.state": state,                             # (B, 19)
    "task": "Pick up the watering can and water the plant in the pot.",
}
action = postprocessor(policy.select_action(preprocessor(batch)))

The preprocessor tokenizes the task string with the google/paligemma-3b-pt-224 tokenizer (max_length=200, right padding), so the language prompt must be supplied under the task key.

Limitations

Trained only on scripted IsaacLab demonstrations for a single task — no real-robot data, no teleoperated data, and no task diversity. Expect a substantial sim-to-real gap.
Only the action expert was trained; the frozen VLM backbone limits how much the policy can generalize to prompts outside the single training instruction.
Trained for ~6.6 epochs over 100 episodes; no held-out validation or success-rate evaluation is reported here.

Files

File	Description
`model.safetensors`	Policy weights (~7.5 GB, bf16)
`config.json`	PI05 policy config
`train_config.json`	Full LeRobot training config used for this run
`policy_preprocessor.json` + `policy_preprocessor_step_2_normalizer_processor.safetensors`	Input pipeline & normalization stats
`policy_postprocessor.json` + `policy_postprocessor_step_0_unnormalizer_processor.safetensors`	Action un-normalization

DexSteer/pi05-isaaclab-ur7e-water-pour

pi05-isaaclab-ur7e-water-pour