LingBot-VA post-trained on Seeed B601 — pick red cube
LingBot-VA post-trained on a single-arm
Seeed B601 pick-and-place task: pick up the red cube and place it inside the tray.
Only the transformer/ is released — the VAE, T5 text encoder and tokenizer are unchanged
from the base model and must be fetched separately (see Setup).
⚠️ This checkpoint has not been evaluated. No real-robot rollout and no held-out split
were run; the training set has 50 episodes of one task and the run does 800 epochs over it,
so expect heavy overfitting to that scene. Treat it as a training artifact, not a policy
with known success rate.
Files
File
What it is
transformer/diffusion_pytorch_model.safetensors
5.09 B params, bf16, 839 tensors
transformer/config.json
WanTransformer3DModel config. attn_mode is set to "torch" for inference — set it to "flex" if you train with this as the init
va_b601_train_cfg.py
The LingBot-VA config this was trained with. Required at inference: it carries the action normalization stats and the channel layout, which are not stored in the checkpoint
norm_stat.json
The same q01/q99 as in the config, standalone for non-LingBot consumers. The .py is authoritative
Copy the layout and action fields from va_b601_train_cfg.py verbatim and add the sampling
parameters (untuned starting points, taken from the RoboTwin/UR3 configs):
Two cameras, resized to 256 x 256 each (the source is 480 x 640, so the aspect ratio is
squashed — do the same at inference), concatenated along width:
Key
observation.images.top
uint8 HxWx3 RGB
observation.images.wrist
uint8 HxWx3 RGB
The prompt must match the training string character for character:
Pick up the red cube and place it completely inside the tray.
Actions: 7 dims, absolute joint positions in degrees, in leader convention, ordered
[shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_yaw, wrist_roll, gripper].
They occupy slots [14, 15, 16, 17, 18, 19, 28] of LingBot-VA's 30-dim action vector; every
other slot is unused. One chunk is frame_chunk_size=4 latent frames x action_per_frame=8
= 32 steps at 30 Hz (~1.07 s).
⚠️ observation.state is not in the same convention as the actions. It is the follower
reading: shoulder_pan, shoulder_lift, wrist_roll and gripper are sign-flipped, and the
gripper uses a different unit (state spans [-270, 0], action [0, 57]). This conversion is
not implemented in the LingBot-VA code — a real-robot client has to do it.
Normalization is min-max over the q01/q99 below, mapped to [-1, 1] and clipped at ±1.5, so a
value far outside this range saturates:
joint
q01
q99
shoulder_pan
-27.10
28.00
shoulder_lift
0.00
131.50
elbow_flex
-109.70
0.00
wrist_flex
0.30
60.60
wrist_yaw
-14.50
7.50
wrist_roll
-9.50
61.30
gripper
0.00
56.50
Training
Post-trained from robbyant/lingbot-va-base (commit 68b7bc1b35da6ddc67ea94c4ceb58d768fbb3f9c).
Data: 50 episodes / 37,932 frames / 1 task, LeRobot v2.1. Videos pre-encoded with the
Wan2.2 VAE at 256 x 256 -> 16 x 16 latents, frame stride 2 over the 30 fps source (15 fps),
84-113 latent frames per episode. One sample is a full episode.
Recipe: 10,000 optimizer steps, batch size 1 per rank, no gradient accumulation
(effective batch 4), AdamW lr 1e-5 (β 0.9/0.95, wd 0.1, 10 warmup steps), grad clip 2.0,
bf16, FSDP + activation checkpointing, 10% text-embedding dropout for CFG. Flow-matching
loss on video and action jointly, snr_shift 5.0 (video) / 0.05 (action); chunk size 1-4
and attention window 4-64 randomized per step.
Compute: 4 x H100 NVL, ~17.6 h (~7 s/step).
Not included
No optimizer/training state — LingBot-VA's save_checkpoint does not write one, so this
checkpoint can be used to initialize training but cannot resume a run.
No VAE / text encoder / tokenizer (identical to the base model, see Setup).
No empty_emb.pt; that is only used for CFG dropout during training. The inference server
encodes the empty string itself.
