MolmoBot-Pi0 — reason-init action policies on real-DROID toys
MolmoBot-Pi0 (π0 / PaliGemma-backbone) action policies fine-tuned on a real-DROID tabletop
pick-and-place set. Each fine-tuned variant is initialized from a different reasoning-only LoRA
baked into the backbone before action training.
subfolder
what it is
full/
reasoning init: all annotation modules
vqa_only/
reasoning init: free-text rationale only
trace_only/
reasoning init: end-effector trace only
merged_alpha0.5/
full/ interpolated halfway back toward the pretrained base
merged_alpha0.5_no_reasoning/
the same interpolation applied to a policy trained without any reasoning stage — the baseline for the merge
Training
Three sequential stages; reasoning is not present at inference:
Reasoning-only LoRA on the backbone, no action loss.
Merge the LoRA into the MolmoBot-Pi0-DROID checkpoint.
Action fine-tune with a flow-matching loss, vision tower frozen.
Action targets are absolute future joint positions (7 arm joints + gripper), horizon 16.
Observations are two cameras, letterboxed to 224×224.
Weight interpolation (merged_alpha0.5/)
RETAIN / WiSE-FT style: the language-model weights are interpolated between the pretrained base
and the fine-tuned policy, while the vision tower and action expert stay fully fine-tuned.
Interpolation traded terminal precision for smoother, better-scaled trajectories in our
evaluation. It is a property of the merge itself and not of the reasoning initialization: the same
effect appears when merging a policy trained without any reasoning stage, which is provided as
merged_alpha0.5_no_reasoning/ so the two can be compared directly at matched alpha.
Contents
Each subfolder contains model.safetensors (flat π0 checkpoint), metadata.pt, and assets/
with the normalization statistics. Optimizer state is not included. The normalizer comes from the
training set's statistics.
Limitations
Research checkpoints from an ablation study, released for reproducibility. They are trained on a
single scene with one robot and are not tuned or benchmarked as a general-purpose policy; the
three reasoning-init arms are not claimed to differ in downstream performance.
