Views
No views yet
video_dim=3072)dim=768, ffn_dim=3072, num_heads=12cross_attn_detach (REPA-style, stop gradient at bridge)0,1,2,...,29)bridge_exclude_full_ref: enabled| Checkpoint | val/loss_action | val/loss_video | val/loss (total) | Notes |
|---|---|---|---|---|
step-3000.safetensors | 0.167 | 0.040 | 0.207 | Best overall validation loss |
step-5000.safetensors | 0.192 | 0.065 | 0.257 | Mid-late training |
step-7000.safetensors | 0.216 | 0.057 | 0.273 | Latest checkpoint |
step-3000 for best action prediction quality.wuji_hand_gesture1# Server side
2python -m deploy.scripts.serve_policy \
3 --checkpoint step-3000.safetensors \
4 --task_name "hand gesture" \
5 --action_dim 20 \
6 --bridge_type cross_attn_detach \
7 --action_dit_num_layers 30 \
8 --action_dit_bridge_layers "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29" \
9 --video_dim 3072 \
10 --num_frames 17 \
11 --port 8000
12
13# Client side
14from deploy.serving import WebsocketClientPolicy, ActionChunkBroker
15
16client = WebsocketClientPolicy(host="gpu-server", port=8000)
17broker = ActionChunkBroker(client, action_horizon=17, replan_steps=5)
18
19result = broker.infer({"head_camera": observation_image})
20action = result["actions"] # (20,) single-step action