Finetune of Isaac GR00T N1.6 for the Galaxea R1 Lite embodiment that outputs end-effector pose
actions instead of arm joint actions, trained with VLASH temporal delay augmentation: for
every sample a delay k ~ U[0, 4] is drawn and the state/action windows are shifted by k while
video and language stay at the base timestep.
video/language: t (stale observation)
state: t + k
action chunk: action[t + k : t + k + horizon]
The policy therefore predicts an action chunk that begins k steps after the observation it
was given, which is what makes asynchronous chunk execution possible on the real robot: the next
chunk can be computed while the current one is still executing.
EE poses are xyz + rot6d (ActionFormat.XYZ_ROT6D), where rot6d is the first two rows of the
rotation matrix — 9 numbers per arm, not a 6-DoF pose vector. EE actions are relative
(ActionRepresentation.RELATIVE, ActionType.EEF) against the corresponding EE state as the
reference frame; grippers are absolute on a 0–100 scale.
Details
Base model
Isaac GR00T N1.6 (nvidia/GR00T-N1.6-3B)
Checkpoint
step 160000
Embodiment tag
new_embodiment (R1 Lite, bimanual + 2 grippers)
Action chunk (training)
32
max_delay_steps
4 (random per sample, k ~ U[0, 4])
use_state_ground_truth
true (delayed state is the measured state at t + k)
Global batch size / steps
32 / 160000
Dataset
R1 Lite foldhoodie, 50 episodes, 72474 frames @ 31 fps
Open-loop evaluation
Delay sweep with gr00t/eval/delayed_open_loop_eval.py, 3 trajectories × 800 steps, horizon 16.
Unnormalized error, averaged over trajectories.
EE pose MSE (metres / rot6d units):
delay
0
1
2
3
4
5
6
7
8
left
5.0e-5
4.4e-5
4.3e-5
4.9e-5
4.8e-5
5.3e-5
6.0e-5
6.8e-5
7.2e-5
right
6.1e-5
6.1e-5
6.3e-5
7.3e-5
7.8e-5
8.5e-5
9.8e-5
1.04e-4
1.13e-4
mean
5.5e-5
5.2e-5
5.3e-5
6.1e-5
6.3e-5
6.9e-5
7.9e-5
8.6e-5
9.2e-5
EE MAE at the minimum is 3.2e-3 (left) / 4.3e-3 (right), i.e. a few millimetres against an xyz
range of roughly 0.3–0.45 m.
EE error bottoms out at delay 1–2, which is the mean of the U[0, 4] training distribution,
and rises monotonically out to delay 8 (about 1.8× the minimum). Gripper error is uncorrelated
with delay — those are near-binary 0/100 transitions where a single mistimed frame swamps the
metric, and the between-trajectory spread (right gripper: ~30 on one trajectory vs ~250 on
another) is far larger than any delay effect. The concatenated 20-dim MSE is dominated by the
grippers for the same reason, so use the EE rows to compare delays.
Caveats: the sweep was run on the training dataset, so these are train-set fit, not held-out
generalization. Also, the EE action labels in this dataset are the measured EE pose at t + 1
(a state-shifted pseudo-action), not the controller's commanded target, so they carry no tracking
error or actuation lag.
Files
Inference artifacts only — optimizer.pt, scheduler.pt, rng_state_*.pth, and the trainer
state are not included, so this checkpoint cannot be used to resume training.
Because of the delay augmentation, the returned chunk is meant to start executing a few steps
after the observation timestamp rather than immediately. The EE outputs are absolute poses in the
same frame as observation.state.*_ee_pose_9d; a downstream IK step is required to drive the
arms, which the joint-action variant
(XYZPIT/vlash-random4-gr00t-n1.6-160000)
does not need.
