This is the initialization state of a MolmoAct2 SO-101 fine-tuning run with Grid Sampler vision-token pruning enabled. It is not a trained policy — it is the exact artifact that lerobot-train holds in memory before its first optimizer step, published so the retraining run is reproducible from step 0.

The Grid Sampler weights in this checkpoint are randomly initialized. This checkpoint is a training seed, not something to evaluate.

MolmoAct2 feeds the VLM one pooled 14×14 patch grid (196 tokens) per camera. Grid Sampler replaces that grid with grid_token_sampler_num_tokens tokens whose locations are predicted rather than fixed: a scout MLP reads the globally pooled feature map and emits K normalized (x, y) coordinates, features are bilinearly sampled at those coordinates, and a coordinate encoder adds a positional embedding. With 2 cameras that is 392 → 32 image tokens at K=16, shortening the prefix the LLM attends over.

use_grid_token_sampler        : false -> true
grid_token_sampler_num_tokens :      -> 16
action_mode                   : continuous -> both
setup_type                    : "single so100/so101 robotic arm in molmoact2"
                                -> "single SO-101 5-DOF robotic arm with gripper"
control_mode                  : "absolute joint pose" (unchanged)

chunk_size and n_action_steps stay at 30. They are not free parameters here: apply_norm_tag_metadata() pins both from the so100_so101_molmoact2 norm-tag metadata in the base checkpoint, and it runs after CLI overrides, so passing a different value has no effect. The SO-101 baseline runs this checkpoint is meant to be compared against were trained the same way.

The token budget is part of the processor config as well as the model config — the preprocessor has to emit the matching number of image placeholder tokens. Both are set consistently in this checkpoint; loading it with a different grid_token_sampler_num_tokens will not work.

Normalizer statistics come from the SO-101 training corpus (see below), not from the base checkpoint's so100_so101_molmoact2 norm tag.

From docs/results_drift_keystone_libero.md in the training repo — LIBERO, 4 suites × 10 tasks × 20 episodes = 800 episodes per model, 10-step flow matching:

K=8 falls off a cliff; K=16 holds parity with the unpruned control at 12× fewer image tokens, and K=32 buys nothing over K=16. Those are LIBERO numbers on a different embodiment — SO-101 parity has not yet been measured, which is what the run seeded by this checkpoint is for. Override with GRID_NUM_TOKENS to sweep.

Merged SO-101 corpus, 177,661 frames from 11 dataset repos across four categories:

By default batches are drawn by uniform shuffling, so composition follows the frame shares above — 2 cm teleop dominates at 62%. A balanced batch sampler that instead pins a fixed 6 : 2 : 3 : 1 mix in every batch is available but opt-in, via CATEGORY_META=<sidecar> (requires batch_size >= 12 for every category to get a slot). Normalizer statistics are computed over the whole corpus and are unaffected by the sampling mode, so this checkpoint is valid for either.