1 Huazhong University of Science and Technology 2 Dongfeng Research & Development Institute † Project lead
🔍 Overview
SimWAM is a World-Action Model for end-to-end autonomous driving. It transfers video-dynamics priors to trajectory prediction by jointly training a pretrained video expert and a lightweight action diffusion transformer with flow matching.
An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference.
This repository contains the official supervised and reinforcement-learning checkpoints for SimWAM on NAVSIM.
Method Summary
SimWAM jointly trains a pretrained video expert and an action expert through a unified attention interface. Both experts attend to the current visual observation, while future-video tokens and action tokens remain isolated from each other.
This design transfers motion and traffic-dynamics knowledge from video prediction to trajectory planning without requiring future-scene generation during deployment. The action expert can subsequently be optimized using FlowGRPO and the compositional NAVSIM PDM reward.
✨ Key Ideas
Joint flow-matching training of a pretrained video expert and a lightweight action diffusion transformer.
Isolated attention between future-video tokens and action tokens.
Direct trajectory prediction without future-video generation at inference time.
Action-only reinforcement-learning fine-tuning with FlowGRPO.
Independent scaling of the video and action experts without shared model parameters.
🎯 Task, Inputs, and Outputs
SimWAM performs end-to-end ego-trajectory planning from a single front-facing camera observation and vehicle-conditioning information.
Because autonomous-driving trajectory planning does not correspond to a standard Hugging Face inference pipeline, this model repository uses pipeline_tag: other.
Benchmark
Inputs
Output
Prediction horizon
Sensor configuration
NAVSIM
Front RGB image, scene-context embedding, 8-D ego state
Normalized future ego trajectory with shape [8, 3]
8 steps
One front-facing camera at 384 × 672
Inputs
The official NAVSIM configuration uses:
One front-facing RGB camera observation resized to 384 × 672
Precomputed text or scene-context embeddings
An 8-dimensional ego-state representation
Model and evaluation configuration supplied by the official SimWAM codebase
Outputs
The action expert predicts:
An eight-step future ego trajectory
Three trajectory values for each predicted step
A tensor with shape [8, 3] for each evaluated sample
The released checkpoints predict normalized trajectory values. The official dataset statistics and NAVSIM evaluation pipeline are required to convert predictions into the benchmark action representation.
📦 Released Checkpoints
Training stage
Checkpoint
Description
Supervised learning
weights/SimWAM.pt
SimWAM checkpoint trained with joint video-action flow matching
Reinforcement learning
weights/SimWAM-RL.pt
SimWAM checkpoint fine-tuned with FlowGRPO and the NAVSIM PDM reward
The released files are custom PyTorch checkpoints. They require the official SimWAM implementation and cannot be loaded directly with transformers.AutoModel.from_pretrained().
🛠️ Usage
You can download the complete model repository using either the Hugging Face CLI or the Python SDK.
NAVSIM, nuPlan/OpenScene data, maps, sensor blobs, precomputed embeddings, and normalization statistics must be prepared according to the official GitHub documentation.
📈 Performance
Using one front-facing camera at 384 × 672, SimWAM reports a 91.5 PDMS score on the NAVSIM navtest benchmark after reinforcement-learning fine-tuning.
Configuration
PDMS
Action-only baseline
86.6
Joint video-action training
90.3
Joint training with FlowGRPO
91.5
For complete evaluation settings and comparisons, refer to the official implementation:
The checkpoints require the official SimWAM implementation and its pinned dependencies.
The model is designed for the NAVSIM data format and evaluation protocol.
Performance outside the training and evaluation distributions is not guaranteed.
The model may produce unsafe or invalid trajectories under unfamiliar road, weather, traffic, or sensor conditions.
Predictions must not be deployed directly on a physical vehicle without extensive validation, safety constraints, fallback systems, and human oversight.
NAVSIM, nuPlan, OpenScene, Wan2.2, and other upstream resources retain their respective licenses and distribution terms.
The SimWAM source repository is released under the MIT License.
Third-party code, datasets, pretrained models, and other upstream assets remain subject to their respective licenses and usage terms.
📖 Citation
If you find SimWAM useful in your research, please consider citing:
bibtex
1@article{zhao2026simwam,
2 title={SimWAM: A Simple World Action Model for End-to-End Autonomous Driving},
3 author={Zongchuang Zhao and Xin Zhou and Tianyang Xu and Zhengyang Sun and Kaixuan Zhou and Honglin Li and Dingkang Liang and Xiang Bai},
4 journal={arXiv preprint arXiv:2608.07468},
5 year = {2026}
6}
