You can deploy this model and start earning money today!

Kennyp-Chen/
starvla-libero-goal-experiments

Name: starvla-libero-goal-experiments
Availability: InStock

Views

No views yet

StarVLA — LIBERO-Goal Experiments

Experiment results on LIBERO-Goal benchmark (10 tasks × 50 trials = 500 episodes per model).

All models use Qwen3.5-0.8B as the VLM backbone, fine-tuned on a single NVIDIA RTX 5090 (32GB).

Results Summary

Model	Action Head	VLM	Steps	Training Time	GPU Memory	Success Rate
OFT	MLP regression	Qwen3.5-0.8B (full FT)	20000	~140h	27,448 MiB	93.0% 🔥
GR00T	Flow-Matching DiT (8 layers)	Qwen3.5-0.8B (full FT)	20000	~26h	23,194 MiB	80.6% ✅

Per-Task Comparison

Task	OFT (93.0%)	GR00T (80.6%)	Gap
turn_on_the_stove	100%	100%	=
put_the_wine_bottle_on_top_of_the_cabinet	100%	100%	=
put_the_wine_bottle_on_the_rack	100%	62%	-38%
put_the_bowl_on_the_stove	100%	96%	-4%
push_the_plate_to_the_front_of_the_stove	98%	52%	-46%
put_the_bowl_on_the_plate	96%	96%	=
put_the_bowl_on_top_of_the_cabinet	94%	96%	+2%
open_the_middle_drawer_of_the_cabinet	94%	96%	+2%
put_the_cream_cheese_in_the_bowl	92%	80%	-12%
open_the_top_drawer_and_put_the_bowl_inside	56%	28%	-28%

Directory Structure

📦 starvla-libero-goal-experiments/
├── README.md
├── OFT_Qwen3.5-0.8B_libero_goal_20K/
│   ├── checkpoint/
│   │   └── steps_20000_pytorch_model.pt
│   └── videos/
│       ├── rollout_<task>_episode<N>_success.mp4  (1 per task)
│       └── rollout_<task>_episode<N>_failure.mp4  (all failures)
└── GR00T_N1.5_Qwen3.5-0.8B_libero_goal_20K_lowmem/
    ├── checkpoint/
    │   └── steps_20000_pytorch_model.pt
    └── videos/
        ├── rollout_<task>_episode<N>_success.mp4  (1 per task)
        └── rollout_<task>_episode<N>_failure.mp4  (all failures)

Training Config Highlights

OFT (93.0%)

Framework: QwenOFT
Action head: 4-layer MLP (512 → 1024 → 1024 → 512 → 7)
Batch size: 4, No gradient accumulation
Learning rate: 3e-4 (cosine schedule)
VLM frozen: No (full fine-tune)

GR00T (80.6%)

Framework: QwenGR00T
Action head: DiT-B (8 layers, 12 heads, 768 hidden)
Action horizon: 8
Repeated diffusion steps: 2
Inference timesteps: 4
Batch size: 4, Gradient accumulation: 4 (effective batch 16)
Learning rate: VLM 1e-5, Action Head 1e-4

Notes

Both models trained on single RTX 5090 (32GB)
DeepSpeed ZeRO-2, BF16 mixed precision
No flash-attn (incompatible with Blackwell + cuDNN 9.20)
GR00T needed gradient accumulation ×4 to fit in 24GB
Full training/eval guide: VLA_WAM_Practice/AGENTS.md

Kennyp-Chen/starvla-libero-goal-experiments