pi05_bread_r1_lora config; BreadInputs/BreadOutputs in
src/openpi/policies/bread_policy.py). Data pipeline + decision docs:
https://github.com/BrianZhengJourney/bread-toaster.10000/params — orbax params (servable with openpi create_trained_policy)10000/assets — norm stats (quantile), computed on the 200 training episodes.
The policy is meaningless without them; loaded automatically from the ckpt dir.| Item | Value |
|---|---|
| Obs images | left_wrist_image, right_wrist_image — uint8 HWC, resize-with-pad → 224×224 (done inside the transform stack). No base/top camera: the base_0_rgb slot is zeroed + masked (training top view was a human egocam; do not feed the RealSense) |
| Obs state | 20D [Lxyz, Lrot6d, Lgrip, Rxyz, Rrot6d, Rgrip], relative_to_first: per arm T_rel = inv(T_arm,start) @ T_arm,t (record each arm's grasp_site pose at rollout start, re-express every subsequent pose). Frame = URDF grasp_site (yam_linear_4310 canonical TCP). Gripper: 0=closed, 1=open |
| Prompt | take the bread out of the bowl and put the bread into the toaster (exact string) |
| Action | (40, 14) @30 Hz: per arm [Δxyz(3, TCP frame), axis-angle Δrot(3), ABSOLUTE gripper target(1)], stepwise-chained: delta_k = inv(T_{t+k-1}) @ T_{t+k}; executor integrates T_cmd,k = T_cmd,k-1 @ delta_k starting from T_current. SO(3) composition only; denormalize exactly once |
| Latency | UMI-style: feed the proprio sampled at image exposure time; start executing the chunk at step ceil(latency × 30 Hz) |
openpi_client; the model serves from a lab
GPU workstation (>=16GB VRAM). See deploy/serve_bread.sh (server, one
command) and deploy/client_example.py (the fixed obs-in -> actions-out
contract for the executor).bread-r1)1from openpi.training import config as _config
2from openpi.policies import policy_config
3
4policy = policy_config.create_trained_policy(
5 _config.get_config("pi05_bread_r1_lora"), "<this-repo-local-path>/10000")
6out = policy.infer({
7 "left_wrist_image": left_uint8_hwc, "right_wrist_image": right_uint8_hwc,
8 "state": rel_state_20d,
9 "prompt": "take the bread out of the bowl and put the bread into the toaster",
10})
11actions = out["actions"][:, :14] # (40, 14)gs://openpi-assets/checkpoints/pi05_base · LoRA gemma_2b_lora + gemma_300m_lora · bs 32 · lr 2.5e-5 cosine · stopped at 10k/20k (plateau; lr was ~50% peak — resume from 10000 if undertrained on-robot)openpi, run r1_top200 (8d6mivlw)