Diffusion Policy (as per Diffusion Policy: Visuomotor Policy Learning via Action Diffusion) trained for the PushT environment from gym-pusht.

See the LeRobot library (particularly the evaluation script) for instructions on how to load and evaluate this model.

The model was trained using LeRobot's training script and with the pusht dataset, using this command:

The training curves may be found at https://wandb.ai/alexander-soare/Alexander-LeRobot/runs/508luayd.

This took about 7 hours to train on an Nvida RTX 3090.

The model was evaluated on the PushT environment from gym-pusht and compared to a similar model trained with the original Diffusion Policy code. There are two evaluation metrics on a per-episode basis:

Here are the metrics for 500 episodes worth of evaluation. For the succes rate we add an extra row with confidence bounds. This assumes a uniform prior over success probability and computes the beta posterior, then calculates the mean and lower/upper confidence bounds (with a 68.2% confidence interval centered on the mean). The "Theirs" column is for an equivalent model trained on the original Diffusion Policy repository and evaluated on LeRobot (the model weights may be found in the original_dp_repo branch of this respository).

The results of each of the individual rollouts may be found in eval_info.json.