You can deploy this model and start earning money today!

kzrzhou/
lingbot-va-libero-object-sft-0522

Name: lingbot-va-libero-object-sft-0522
Availability: InStock

robbyant/lingbot-va-base

finetune

other

Views

No views yet

LingBot-VA SFT on LIBERO-Object (2026-05-22)

Five SFT checkpoints of robbyant/lingbot-va-base post-trained on the LIBERO-Object benchmark (10 tasks × 10 demos = 100 demos), plus a 4-task warm-start fine-tune that lifts all 10 tasks to nonzero SR. Per-trial evaluation artifacts and the full reproducibility documentation included.

Results (triage eval, 5 trials × 10 tasks = 50 trials, MAX_STEPS=300)

ckpt	mean SR	per-task (alphabet soup / cream cheese / salad dressing / bbq / ketchup / tomato / butter / milk / pudding / orange juice)
`checkpoint_step_1000`	18.0% (9/50)	0/40/0/20/20/40/0/0/20/40
`checkpoint_step_2000`	34.0% (17/50)	40/60/60/40/40/20/0/0/0/80
`checkpoint_step_3000`	48.0% (24/50)	80/100/80/40/60/40/0/20/20/40
`checkpoint_step_2500_extra`	52.0% (26/50)	60/100/80/60/40/40/40/0/40/60
`checkpoint_step_3000_extra`	66.0% (33/50)	80/80/40/80/60/80/60/60/100/20

checkpoint_step_3000_extra is the overall winner — all 10 tasks reach nonzero SR, with butter going 0→60%, milk 20→60%, and chocolate pudding 20→100%. *_extra checkpoints come from a 1000-step warm-start fine-tune that loads checkpoint_step_2000 weights and trains on just 40 demos covering the 4 weakest tasks (tomato sauce, butter, milk, chocolate pudding). See REPRODUCE.md §8 for full details, including the trade-off (mild regressions on tasks not in the fine-tune subset, notably orange juice 40→20% and salad dressing 80→40%).

If you care about uniform task coverage, checkpoint_step_3000_extra is the right pick. If you want the best balanced model from the main run, checkpoint_step_3000 is the choice (4 tasks ≥ 80% but butter at 0%).

Repo layout

checkpoint_step_{1000,2000,3000}/         # main run, transformer weights only (config.json + safetensors)
checkpoint_step_{2500,3000}_extra/        # warm-start fine-tune, transformer weights only
outputs/                                  # per-trial mp4s, per-task JSONs, index.html dashboard
REPRODUCE.md                              # full reproducibility guide
RETROSPECTIVE.md                          # post-mortem of the prior failed (IPEC) run

To use a checkpoint, you also need the VAE + text encoder + tokenizer from the base model — symlink them in from robbyant/lingbot-va-base:

python

from huggingface_hub import snapshot_download
ckpt = snapshot_download("kzrzhou/lingbot-va-libero-object-sft-0522",
                         allow_patterns=["checkpoint_step_3000_extra/**"])
base = snapshot_download("robbyant/lingbot-va-base",
                         allow_patterns=["vae/**", "text_encoder/**", "tokenizer/**"])
# Then point va_libero_cfg.wan22_pretrained_model_name_or_path at a dir that
# combines the transformer/ from `ckpt` with vae/text_encoder/tokenizer from `base`.

Training summary (full details in REPRODUCE.md)

Hardware: 1× A100 80 GB
Data source: raw LIBERO HDF5 (yifengzhu-hf/LIBERO-datasets/libero_object/*.hdf5), not the IPEC pre-converted mirror (which causes 0% SR — see RETROSPECTIVE.md)
Subset: 10 demos/task × 10 tasks (seed-42 random sample)
Optimizer: bitsandbytes.optim.AdamW8bit, lr 1e-5, 10-step warmup
Effective batch: 10 (batch_size=1 × grad_accum=10)
3000 steps, ~8h40m wall, ~10.4 s/step, peak 34.5 GB GPU memory
Optional warm-start: +1000 steps on 4-task / 40-demo subset (~3h, see REPRODUCE.md §8)