VERA checkpoints
Hosted weights for VERA — Turning Video Models into Generalist Robot Policies — a two-stage
system: a video planner that "dreams" the future, and a Jacobian inverse-dynamics model (IDM)
that turns the dream into robot actions. VERA hosts only the trained artifacts; frozen upstream pieces
(Wan2.1 text-enc/VAE, VGGT) are pulled from their original homes and filtered out at inference.
Code + usage: see the VERA GitHub repo.
Release groups
1 · Panda-sim (MimicGen)
| dir | what |
|---|
mimicgen-wan-1.3b/ | MimicGen-specialist WAN video planner (1.3B, t2v→v2v), DiT-only bf16 (~2.8 GB, de-bloated from 17.5 GB) + flow_decoder.ckpt + algo_config.yaml. Pull Wan-AI/Wan2.1-T2V-1.3B base upstream. |
idm-mimicgen-285ouq1q/ | Jacobian IDM (run 285ouq1q, VGGT-1B) used for the reported MimicGen stack_d0 evaluations; pairs with mimicgen-wan-1.3b/. |
Pair these two + cotracker + the gated/adaptive gripper controller to run stack_d0.
2 · PushT — the two-stage DFoT→Jacobian pipeline
| dir | what |
|---|
pusht-dfot/ | DFoT video planner (run dvxixf6d) — a U-Net3D flow predictor (~2.4M params), not WAN. + run_config.yaml. |
pusht-idm/ | PushT Jacobian IDM (run j1j59qzz, ~34.8M params, 2-DOF) + config.yaml. |
3 · OMNI — the cross-embodiment WAN planner
| dir | what |
|---|
omni-wan/ | OMNI WAN planner (14B i2v→v2v, combined_4env), DiT-only bf16 (~33 GB). Pull Wan-AI/Wan2.1-I2V-14B-480P base upstream. |
idm-mimicgen/ | MimicGen IDM (x21o0cwe) — the OMNI-default pairing. |
idm-droid/ | DROID FR3 IDM (7wohna95, SE3-delta 7-DOF). |
4 · DROID — real-robot WAN planner
| dir | what |
|---|
wan-droid-14b/ | DROID-specialist WAN video planner (14B i2v→v2v, 29 context frames → 24-frame chunks, 3-view canvas), DiT-only bf16 (~31 GB) + algo_config.yaml. Pull Wan-AI/Wan2.1-I2V-14B-480P base upstream. Used by examples/droid_generation.ipynb in the GitHub repo — language-conditioned video generation from real context frames (no server or robot needed). |
droid-demo-clips/ | Real DROID context clips (~100 MB) loaded by the generation notebook. |
Licenses (per checkpoint)
VERA's own contributions (WAN video planners, Jacobian IDM heads, DFoT planner) are Apache-2.0.
Two IDM checkpoints additionally
bundle the VGGT-1B backbone weights, which are
CC-BY-NC-4.0 — so those combined artifacts are non-commercial.
| dir | license | why |
|---|
wan-droid-14b/, omni-wan/, mimicgen-wan-1.3b/ | Apache-2.0 | WAN planners; inherit Wan2.1 Apache-2.0 |
pusht-dfot/, pusht-idm/ | Apache-2.0 | authors' weights; VGGT not bundled |
idm-mimicgen/ | Apache-2.0 | authors' IDM head only; pulls VGGT-1B upstream at runtime (itself CC-BY-NC-4.0) |
idm-droid/, idm-mimicgen-285ouq1q/ | CC-BY-NC-4.0 | bundle VGGT-1B weights in-checkpoint |
Frozen upstream pieces (Wan2.1 text-enc/VAE, VGGT-1B) keep their original licenses.
Training data — pusht-packed/ and pusht-noise-packed/
Two packed PushT training sets, same on-disk format (one NPZ per episode with 504×504 JPEG frames,
MegaFlow optical flow qint8_zstd_npz-quantized, and per-episode state, plus index.json /
format_manifest.json; per-folder provenance in provenance.json, file hashes in SHA256SUMS.txt):
| Folder | Episodes | Size | Content | Actions |
|---|
pusht-packed/ | 206 | ~1.9 GB | human-teleop episodes — the Jacobian IDM training set (config_pusht_vggt_fusion_jacobian) | command actions from pusht-replay/, the original pusht_cchi_v7_replay.zarr archive |
pusht-noise-packed/ | 18,685 | ~61 GB | uniform-random exploration episodes — IDM pretraining / data-efficiency ablations | self-contained: state deltas from the embedded traj_state arrays (no external file) |
1hf download sizhe-lester-li/VERA --include "pusht-packed/*" --local-dir ./vera-ckpts
2hf download sizhe-lester-li/VERA --include "pusht-noise-packed/*" --local-dir ./vera-ckpts # ~61 GB
The standard pack also needs the original command-action replay, now mirrored in
pusht-replay/ (31 MB, 206 episodes). Download, verify and extract:
1hf download sizhe-lester-li/VERA --include "pusht-replay/*" --local-dir ./vera-data
2(cd vera-data/pusht-replay && sha256sum -c SHA256SUMS.txt)
3unzip -n vera-data/pusht-replay/pusht.zip -d data
Set dataset.pusht_zarr_root to the absolute path of data/pusht.
License note: the
pusht-packed source episodes are the MIT-licensed PushT data from the
Diffusion Policy benchmark; the
pusht-noise-packed source trajectories are the PushT exploration data from
DINO-WM (Zhou et al., 2024). Both were re-rendered with
gym-pusht; optical-flow annotations were added with
MegaFlow.
Regeneration reference:
scripts/data/pack_pusht.py in the VERA repo (walkthrough:
docs/DATA_GENERATION.md).
