WEAVER fine-tuned on RoboLab (Marlowe full-data run) — model card
Fine-tune of the released WEAVER action-conditioned multi-view world model on Pi0.5 RoboLab demos,
trained on Stanford's Marlowe H100 cluster (plan 011). This is the clean re-encode + full-data run
that plan 009 could not complete on GCP (a full 250 GB disk killed 3 of 12 encode batches there).
Training data
- 1,045 train + 55 val episodes from 29 RoboLab tasks (the 92 non-empty raw HDF5 files of the
original 178-file collection; 86 were empty stubs — see the plan-011 audit).
- Encoded once to SD3 latents + CLIP text features on Marlowe, 4-way GPU-sharded,
--ext1 left
(over_shoulder_left in panel 0, aligned with the Pi0 policy's exterior_image_1_left).
- The
val split (55 episodes) is frozen (val_ids.txt) so any future rung shares it byte-for-byte.
Training
- 4× H100 DDP (torchrun), global batch 24 (per-GPU 6), 16,000 steps, LR 2e-5, compile off, seed 0.
- ~2.6 h wall at ~0.57 s/step.
Final validation (step 16,000)
| Camera | FVD ↓ | FID ↓ | LPIPS ↓ |
|---|
| exterior_1_left | 136.4 | 29.0 | 0.070 |
| wrist_left | 383.5 | 52.8 | 0.240 |
(Wrist is the harder view. PSNR/SSIM not enabled in this eval config.)
⚠️ Reward / critic heads are a PLACEHOLDER — not usable for planning (yet)
The exporter wrote a success-gated linear ramp into reward_progress, not RoboMeter (plan 010), and
this run did not freeze the base heads. So the reward and critic heads learned a clock, not a real
value signal — do not use them for best-of-N or value-based planning. The video-prediction metrics above
are unaffected (driven by the Flow loss). A RoboMeter-labelled reward/critic fine-tune is the planned
follow-up (plan 010/011).
