ACT — dump_bin_interrupt_real (real-rig calibrated xArm7)
Action Chunking Transformer trained on Genesis HR Bench's real-rig calibrated
dump_bin_interrupt_real task: an xArm7 in the identity frame of the physical rig, with the
RealSense D455 head-camera extrinsics and hand-eye wrist transform taken from
the 2026-08 calibration (real/sim_setup.py).
Inputs / outputs
| |
|---|
observation.image | 256x256x3 — head camera (D455, world pose (0.925, -0.267, 0.293)) |
observation.wrist_image | 256x256x3 — right wrist camera, attached to link7 |
observation.state | 8-D [7 arm qpos, gripper], gripper 1.0 = open |
action | 8-D [7 commanded target qpos, gripper] — absolute, not deltas |
Action space is qpos_target_abs: the recorded PD targets, not measured qpos and
not deltas. Serve it with --action-type qpos_abs.
Training
| |
|---|
| Data | 300 episodes, runs/real/vla_data_0808/dump_bin_interrupt_real (scripted rollouts, rasterizer) |
| Steps | 25,000 (batch 80, 27.2 epochs) |
| Optimizer | AdamW, lr 1e-05, wd 0.0001, grad clip 10.0 |
| Chunk | chunk_size=100, n_action_steps=1, temporal_ensemble_coeff=0.01 |
| VAE | disabled (use_vae=false) |
| Normalization | MEAN_STD for visual / state / action |
| Final loss | 0.028 |
| Hardware | 1x H100 80GB, 1h12m |
Usage
1# serve (lerobot v0.1.0 venv — this is a v0.1.0-format checkpoint:
2# config.json + model.safetensors, no processor files)
3python baseline/servers/lerobot_server.py \
4 --policy-type act --checkpoint <local_dir> --port 8775
5
6# evaluate
7python scripts/eval_vla.py --task dump_bin_interrupt_real --model act \
8 --server-url http://127.0.0.1:8775 --action-type qpos_abs
Status — read before using the benchmark number
The weights reproduce their training targets closely. Replaying the training
episodes' own observations through the served policy (scripts/check_act_replay.py,
frame-by-frame):
| joint MAE | gripper MAE | gripper range predicted |
|---|
| with temporal ensembler | 0.012-0.014 rad | 0.003 | 0.377 - 1.00 |
| single-shot (ensembler cleared each step) | 0.012-0.017 rad | 0.004 | 0.375 - 1.01 |
(ground-truth gripper is 0.375 closed / 1.0 open, so the closed command is
being reproduced correctly.)
However, in-simulator benchmark success is currently 0% — and that is an
eval-harness bug, not a property of these weights. In the eval path the human
avatar is left at its home pose, which sits directly in the calibrated head
camera's field of view: the policy sees a torso where the robot arm should be,
from step 0 onward. Re-running the collection pipeline on the same seed
produces a clean view (the avatar's forearm only enters frame during the
interrupt), so the training data is unaffected. Until that is fixed, do not
read 0% as a capability measurement.
Provenance
- Task registry:
real/tasks.py -> dump_bin_interrupt_real
- Training run: SLURM job
62715183, exp act_dump_bin_interrupt_real_20260524_qpos_target_abs_300ep_real0808_c100_s25000_novae_v1
- W&B project:
act_genesis_hr_bench_real_0808 (entity multi-agent-world-model)
- Source dataset (LeRobot):
ghb/dump_bin_interrupt_real_qt24_300_c100
