Views
No views yet
square_d0DinoDynamics, same as dino_step_60000.pt) trained on
MimicGen square_d0. Predicts DINOv2 patch features of the next frame, not pixels:{I_{t-8}, I_t}^{agentview, eye_in_hand}, a[t:t+8], s_t -> DINOv2 features of {I_{t+8}}^V| component | shape | role |
|---|---|---|
| DINOv2 ViT-S/14 encoder | frozen | 2 views → 196 patches × 384 (14×14 grid at 196 px) |
predictor | 6 × TransformerEncoderLayer, d=384, FFN 2048 | the world model proper (~10.6 M) |
act_tok | Linear(56 → 384) | the 8×7 action chunk as one token |
state_tok | Linear(9 → 384) | proprioception as one token |
pos_emb / view_emb / frame_emb | (196,384) / (2,384) / (3,384) | patch, camera, frame-slot identity |
query | (196, 384) | learned queries that read out the next frame |
frame_emb has 3 slots — two history frames plus the query frame. Forward returns
(B, 2, 196, 384): the transformer sees 2 frames × 2 views × 196 context tokens + 2
conditioning tokens + 2 × 196 queries = 1,178 tokens, and only the query tail is returned.residual=False, matching the published checkpoint's configuration.
(to_latent is not part of the world model — it is a feature→SD3-VAE-latent head for pixel metrics,
built only when a VAE is passed and trained on detached features.)| value | |
|---|---|
train feat_mse (final) | 0.563 |
val feat_mse (best) | 0.729 |
val copy_feat_mse (baseline) | 1.472 |
| best ratio | 0.495 |
copy_feat_mse is the error of simply reusing the current frame's features, so the model explains
about half of frame-to-frame feature motion. It starts worse than copying (val 1.667, ratio
1.13) and crosses below the baseline early.threading_d0: different
task, different predictor, 196 vs 256 patches, 196 vs 224 px input. Those numbers are not
commensurable.square_d0:..._40k_failure_seg — tail cut at the outcome, approach kept..._40k_success..._224x224 — 1000 expert demoshistory=2, dino_size=196, state_dim=9 (eef_pos 3 + eef_quat 4 + gripper_qpos 2), horizon 8.dino_step_{5000,10000,15000,20000}.pt and resume_last.pt. Use step 20000; earlier steps are
kept because the metric plateaus, so a shorter run is nearly as good if compute matters.