DiLA is a disentangled latent-action world model trained from observation-only videos. It separates video features into a structure pathway for dynamics-relevant spatial layouts and latent actions, and a content pathway for appearance, texture, and slowly revealed scene details. Future states are predicted by rolling out latent actions in structure space and fusing the predicted structure with content memory.

The released checkpoints are:

text

model.pt
  Recommended by default. Trained on SSv2, RT-1 / fractal20220817_data,
  RECON, and LoopNav, following the setting in the paper.

humanrobot_autoregressive.pt
  For reproducing the human/robot experiments and table results.
  Trained only on SSv2 and RT-1 / fractal20220817_data.

navigation_autoregressive.pt
  For reproducing the navigation experiments.
  Trained only on RECON and LoopNav.

The checkpoint uses the public codebase naming scheme, including structure_encoder.* and content_fusion.* keys.

Paper

DiLA: Disentangled Latent Action World Models
Tianqiu Zhang*, Muyang Lyu*, Yufan Zhang, Fang Fang, Si Wu
ICML 2026

Paper: https://arxiv.org/abs/2605.15725
Project page: https://disentangled-latent-action-world-models.github.io/

Code

The training and evaluation code is available at:

https://github.com/senngadaisuki/disentangled-latent-action-world-models

Usage

Please see the GitHub repository for environment setup, pretrained RAE preparation, checkpoint loading, training, and evaluation instructions.

For interactive qualitative evaluation, the code repository provides test.ipynb, which includes autoregressive generation, action transfer, and rebinding examples on SSv2, RT-1, RECON, and LoopNav.

Training Data

Following the paper, DiLA is trained on observation sequences from:

Something-Something-V2 (SSv2)
RT-1 / fractal20220817_data
RECON
LoopNav

Third-party datasets are subject to their own licenses and terms of use.

Intended Use

The checkpoint is intended for research on latent action models, video prediction, representation learning, action transfer, content-structure disentanglement, and visual planning.

Citation

bibtex

@inproceedings{zhang2026dila,
  title     = {{DiLA}: Disentangled Latent Action World Models},
  author    = {Zhang, Tianqiu and Lyu, Muyang and Zhang, Yufan and Fang, Fang and Wu, Si},
  booktitle = {Forty-third International Conference on Machine Learning},
  year      = {2026},
  url       = {https://openreview.net/forum?id=BRBHruBDkb}
}