NVIDIA GR00T N1.7-3B vision-language-action model, fine-tuned on 5000 scripted Z1 arm grasp episodes to drive the 6-DOF Unitree Z1 arm to pick a small cube off a table.

This is the V1 (state-only) variant: video modality is plumbed end-to-end but fed zero arrays during training (GR00T_SKIP_VIDEO_DECODE=1). The model conditions on 6-DOF Z1 joint state + language. A vision-conditioned successor (V2) is trained on real rendered video — see "Successor".

Highlights

Base model: nvidia/GR00T-N1.7-3B (3.14 B params, AlternateVLDiT diffusion head)
Fine-tuned with: --tune-projector --tune-diffusion-model --no-tune-llm (no --tune-visual)
5000 episodes × 80 frames (LeRobot v2.1)
5000 fine-tuning steps, batch 32
Embodiment tag: EmbodimentTag.NEW_EMBODIMENT (custom modality config)
Action: z1_eef_delta (6) + z1_gripper (2) — relative joint deltas

Verified results

Setup	Cube max lift	Success
Teacher-forced state, kinematic magnet gripper	0.798 m	✅
Fully autoregressive state, kinematic magnet	0.738 m	✅
Inside `warehouse.usd` + PhysX FixedJoint gripper	0.134 m	✅ (max_lift > 0.05 m)

Files

checkpoint-5000/ — final HF Trainer checkpoint folder (config + model shards + processor + experiment_cfg)
Loadable via Gr00tPolicy(...) from the Isaac-GR00T SDK

Usage

python

import sys
sys.path.insert(0, "/path/to/Isaac-GR00T")
import modality_config_z1  # registers EmbodimentTag.NEW_EMBODIMENT

from gr00t.policy import Gr00tPolicy
from gr00t.data.embodiment_tags import EmbodimentTag

policy = Gr00tPolicy(
    embodiment_tag=EmbodimentTag.NEW_EMBODIMENT,
    model_path="m3/go2z1-grasp-gr00t-n17-v1/checkpoint-5000",
    device=0,
    strict=False,
)

obs = {
    "state":    {"z1_joint_pos": z1_joint_pos[None, None, :]},  # (1, 1, 6)
    "video":    {vk: zeros((1, 1, 64, 64, 3), dtype=uint8)
                 for vk in policy.modality_configs["video"].modality_keys},
    "language": {"annotation.human.task_description": [["pick up the cube"]]},
}
action, _ = policy.get_action(obs)
# action keys: 'z1_eef_delta' (6), 'z1_gripper' (2)

For a full closed-loop demo with autoregressive inference + PhysX-constraint gripper, see stage4_joint_eval/gr00t_warehouse_physical_grasp.py.

Training data

V1 was trained with dummy zero-array video (state + language only). The dataset structure (LeRobot v2.1 with 64×64 placeholder mp4s):

Layout: meta/info.json, meta/episodes.jsonl, meta/modality.json, data/chunk-XXX/episode_*.parquet, videos/chunk-XXX/{base_cam,wrist_cam}/episode_*.mp4
Generation script: go2_z1_warehouse/stage2_grasp_data/json_to_lerobot.py
Repo: https://github.com/aws300/go2_z1_warehouse

The fully rendered visual successor dataset (224×224, 5000 ep × 2 cams) is published at:

Vision dataset: https://huggingface.co/datasets/m3/go2z1-grasp-vision-v2

Citation

bibtex

@misc{go2z1-grasp-gr00t-n17-v1,
  title  = {Z1 Grasp GR00T N1.7-3B Fine-tune V1 (state-only)},
  author = {m3},
  year   = {2026},
  url    = {https://huggingface.co/m3/go2z1-grasp-gr00t-n17-v1}
}

Successor

V2 (vision-conditioned, trained on real rendered video at 224×224) is being trained: m3/go2z1-grasp-gr00t-vision-v2 (see V2 dataset above).