openpi pi05_bi checkpoint from a bimanual two-tube pick-and-place run with
tactile inputs. This is the step-6000 checkpoint, which had the lowest held-out
validation loss of the run up to step 8000.
Task
Single unified instruction for every episode:
Firstly, use the left hand to pick up the blue tube, and then use the right
hand to pick up the green tube. Next, use the left hand to place the blue tube
back firstly, and then use the right hand to place the green tube back.
two_tubes_02 shipped with the placeholder string "perform manipulation task"
in its meta/tasks.jsonl. Since prompt_from_task=True feeds this string
straight to the model, the two sources were forced onto the single instruction
above at merge time -- otherwise the model would be taught that two different
instructions mean the same motion.
Data
source
episodes
frames
KaiyueChen/two_tubes_01
519
425,115
KaiyueChen/two_tubes_02
500
377,604
merged
1,019
802,719
LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet
files (total_videos=0). Six camera streams: camera0, camera1, and four
tactile sensors (tactile_left_0/1, tactile_right_0/1).
Split
Episodes are held out per source repo (10%, seed 42) so the held-out set
keeps the same source mix as train:
split
episodes
train
917
val_seen (subset of train)
102
val_unseen (held out)
102
Normalization statistics (quantile q01/q99) are computed over the train split
only.
(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule; the peak_lr=2e-4 / decay_steps=100000 block in config.py is referenced only by pi05_single*)
LoRA
rank 16 on the LLM, rank 32 on the action expert
vision tower
fully fine-tuned -- the freeze filter matches only .*llm.*
Validation curve
Flow-matching loss, 20 batches per split, evaluated on the same leading batches
each time so successive points are comparable.
step
train
val_seen
val_unseen
gap
0
0.5525
0.4968
0.5261
0.0293
2000
0.0504
0.0504
0.0608
0.0104
4000
0.0475
0.0467
0.0576
0.0109
6000
0.0448
0.0437
0.0543
0.0105
8000
0.0439
0.0423
0.0550
0.0127
val_unseen fell monotonically through step 6000, then ticked up by 0.0007 at
step 8000 while val_seen kept improving and the gap widened. Each validation
pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so a
change of that size is within noise -- whether step 6000 is the true optimum is
not yet settled.
Contents
checkpoint/
params/ # inference weights
train_state/ # optimizer state, for resuming
assets/two_tubes_0102/
norm_stats.json # computed over the train split only
