pi0-omy-pnp

Task

• Task: Place the red or blue mug on the plate, driven by language instruction
  • "Place the red mug on the plate."
  • "Place the blue mug on the plate."
• Observation: agent camera + wrist camera (256×256) + 6-DoF joint state
• Action: 7-dim (6 joints + gripper)
• Dataset: 20 teleoperated demos (2621 frames, 20fps)

Training

• Policy: PI0 (PaliGemma + action expert), chunk size 50
• Steps: 20,000 (checkpoints at 10k / 20k)
• Hardware: AMD GPU (ROCm 7.x), ~0.83 s/step
• Framework: lerobot (source build, commit 10b7b35), transformers 4.57.6

Closed-loop evaluation (held-out seeds)

Files

• config.json — model configuration
• model.safetensors — trained weights (~7.5 GB)
• train_config.json — training configuration

Usage