You can deploy this model and start earning money today!

Harrysunshine/
act-libero-goal

Name: act-libero-goal
Availability: InStock

Views

No views yet

ACT — LIBERO-Goal "put the bowl on the plate"

An ACT (Action Chunking Transformer) policy trained with LeRobot on a single LIBERO-Goal task: put the bowl on the plate (task suite libero_goal, task_id = 8).

It reaches 100% success rate (10/10 episodes) on that task in the LIBERO simulator.

This checkpoint was produced as a teaching demo for a robot-learning course: it is small (51.7M parameters), trains in minutes on one GPU, and is meant as a clean, reproducible reference point for "what a working single-task imitation policy looks like".

Model details


Policy	ACT (CVAE + action chunking), `lerobot` `type: act`
Parameters	51,671,687 (fp32, 234 tensors)
Vision backbone	ResNet-18, initialised from `ResNet18_Weights.IMAGENET1K_V1`
Transformer	`dim_model=512`, 8 heads, 4 encoder layers, 1 decoder layer, FFN 3200, dropout 0.1, post-norm
CVAE	`use_vae=true`, latent dim 32, 4 VAE-encoder layers, `kl_weight=10.0`
Action chunking	`chunk_size=100`, `n_action_steps=100`, no temporal ensembling
Observation steps	1
Normalisation	mean/std for images, state and actions (`use_imagenet_stats=false`)

Inputs / outputs

Key	Shape	Meaning
`observation.images.image`	`3 × 256 × 256`	agentview (third-person) RGB
`observation.images.image2`	`3 × 256 × 256`	`robot0_eye_in_hand` (wrist) RGB
`observation.state`	`8`	proprioceptive state
`action` (output)	`7`	6-DoF end-effector delta + gripper, relative control

Training

Data: lerobot/libero, restricted to the 49 episodes belonging to put the bowl on the plate in the libero_goal suite.
Optimiser: AdamW, lr 1e-5 (same lr for the backbone), weight decay 1e-4.
Batch size 256, seed 1000, video_backend=pyav.
Released checkpoint: the best-scoring evaluation checkpoint, at step ≈ 950 (≈ 53 epochs over the 49-episode subset, 243,200 samples seen).

Evaluation

Evaluated in the LIBERO simulator on the same task it was trained on (libero_goal, task_id = 8, obs_type=pixels_agent_pos, 256×256 observations):

Metric	Value
Success rate (`pc_success`)	100 (10 / 10 episodes)
Average summed reward	1.0
Step	950

The raw evaluation record is included as eval_summary.json.

Usage

Requires LeRobot with the LIBERO environment (lerobot[libero]). Closed-loop rollout in simulation:

python

import torch
from lerobot.configs.policies import PreTrainedConfig
from lerobot.envs.configs import LiberoEnv as LiberoEnvConfig
from lerobot.envs.factory import make_env, make_env_pre_post_processors
from lerobot.envs.utils import add_envs_task, preprocess_observation
from lerobot.policies.factory import get_policy_class, make_pre_post_processors
import lerobot.policies  # register policy types

policy_path = "Harrysunshine/act-libero-goal"
task_suite = "libero_goal"
task_id = 8  # "put the bowl on the plate"
device = "cuda"

policy_cfg = PreTrainedConfig.from_pretrained(policy_path)
policy_cfg.device = device
policy = get_policy_class(policy_cfg.type).from_pretrained(
    policy_path, config=policy_cfg, strict=False
)
preprocessor, postprocessor = make_pre_post_processors(
    policy_cfg, pretrained_path=policy_path
)

env_cfg = LiberoEnvConfig(
    task=task_suite,
    task_ids=[task_id],
    obs_type="pixels_agent_pos",
    observation_height=256,
    observation_width=256,
    episode_length=300,
)
env = make_env(env_cfg, n_envs=1)[task_suite][task_id]
env_preprocessor, env_postprocessor = make_env_pre_post_processors(env_cfg, policy_cfg)

policy.reset()
observation, _ = env.reset(seed=[7])
for _ in range(300):
    batch = preprocess_observation(observation)
    batch = add_envs_task(env, batch)
    batch = env_preprocessor(batch)
    batch = preprocessor(batch)
    with torch.inference_mode():
        action = policy.select_action(batch)
    action = postprocessor(action)
    action = env_postprocessor(action)
    observation, _, terminated, truncated, info = env.step(action.cpu().numpy())
    if terminated[0] or truncated[0]:
        break

Limitations

Single task, no language conditioning. ACT regresses actions from pixels and state only; this checkpoint has only ever seen put the bowl on the plate and will attempt that behaviour regardless of what else is in the scene.
Simulation only. Trained and evaluated entirely in LIBERO; it has not been deployed on physical hardware and the action space (relative EE control, LIBERO conventions) is simulator-specific.
Observation format is fixed. Two 256×256 cameras (agentview + wrist) and an 8-d state vector, in exactly that layout.
Small evaluation set. 100% is over 10 evaluation episodes from LIBERO's initial-state set, not a large-scale benchmark run.

Citation

ACT / Action Chunking with Transformers:

bibtex

@inproceedings{zhao2023act,
  title     = {Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware},
  author    = {Zhao, Tony Z. and Kumar, Vikash and Levine, Sergey and Finn, Chelsea},
  booktitle = {Robotics: Science and Systems (RSS)},
  year      = {2023}
}

LIBERO benchmark:

bibtex

@inproceedings{liu2023libero,
  title     = {LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning},
  author    = {Liu, Bo and Zhu, Yifeng and Gao, Chongkai and Feng, Yihao and Liu, Qiang and Zhu, Yuke and Stone, Peter},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2023}
}

中文说明

这是一个用 LeRobot 训练的 ACT（动作分块 Transformer） 策略，任务是 LIBERO-Goal 套件中的第 8 个任务 “把碗放到盘子上”（put the bowl on the plate）。

数据：lerobot/libero 中属于该任务的 49 条示范轨迹。
模型：ResNet-18 视觉主干 + 4 层编码器 / 1 层解码器 Transformer + CVAE（隐变量 32 维），一次预测 100 步动作块，51.7M 参数。
输入：第三人称相机 + 腕部相机各一路 256×256 RGB，8 维本体状态；输出 7 维动作。
评测：在 LIBERO 仿真同任务上 10 条 episode 成功率 100%。
定位：机器人学习课程的教学 demo——足够小、训练几分钟就能复现，用来演示“一个能跑通的单任务模仿学习策略长什么样”。

限制：只会这一个任务、不吃语言指令、只在仿真里验证过、观测格式固定（两路 256×256 相机

8 维状态），100% 是 10 条 episode 的结果而非大规模 benchmark。