SAM2Act is a multi-view robotics transformer policy for robotic manipulation. Built on RVT-2, it combines multi-resolution upsampling with visual embeddings from the SAM2 foundation model to improve 3D action prediction, multitask learning, and generalization. SAM2Act+ extends this policy with a memory bank, memory encoder, and memory attention so the agent can condition on prior observations and actions for spatial memory-dependent tasks.

For full project details, code, training instructions, and videos, see the SAM2Act website and GitHub repository.

This model repository stores released SAM2Act and SAM2Act+ checkpoints together with the config files needed for evaluation.

The repository is organized as follows:

sam2act_rlbench/ contains the RLBench checkpoint model_89.pth and its exp_cfg.yaml and mvt_cfg.yaml files. Each MemoryBench task folder contains the Stage 1 checkpoint model_9.pth, the Stage 2 memory-conditioned checkpoint model_plus_19.pth, and the corresponding exp_cfg*.yaml and mvt_cfg*.yaml files.

For evaluation commands and expected directory placement, see the SAM2Act GitHub README or the SAM2Act project website.

If you use these models, please cite the SAM2Act paper: