Model Card for OpenVLA-7B Stretch LoRA
This model is a Low-Rank Adaptation (LoRA) fine-tune of the openvla/openvla-7b Vision-Language-Action (VLA) model, specifically adapted for the Hello Robot Stretch platform. It translates natural language instructions and RGB image observations into continuous 7-Degree-of-Freedom (7-DoF) physical robot actions.
Model Details
Model Description
This model adapts the general-purpose OpenVLA-7B base model to the specific kinematics and workspace of a Hello Robot Stretch. It employs a parameter-efficient fine-tuning (PEFT) strategy, training only ~33.5 million parameters (0.44% of the base model) via LoRA. The model predicts normalized continuous actions in a 7-DoF space based on user instructions and visual input.
- Developed by: Yashwanth Gowda
- Model type: Vision-Language-Action (VLA) Model (LoRA Adapter)
- Language(s) (NLP): English
- License: MIT
- Finetuned from model:
openvla/openvla-7b
Uses
Direct Use
The model is designed for direct zero-shot inference on a physical Hello Robot Stretch. It accepts a 640x480 RGB image observation and a text instruction (e.g., "pick up the water bottle") and autoregressively generates action tokens. These tokens are detokenized into continuous physical actions for the robot's base, arm, and gripper.
Out-of-Scope Use
This model is strictly fine-tuned for the physical capabilities of the Hello Robot Stretch. It is not intended for use with high-payload industrial arms (e.g., UR5, Franka) or quadrupedal robots without further domain adaptation.
Training Details
Training Data
The model was fine-tuned on a custom behavioral cloning dataset collected via teleoperation on the Hello Robot Stretch platform[cite: 3].
- Total Episodes: 52
- Total State-Action Transitions: 10,652
- Action Space: Natively 6-DoF (
arm_extend, lift_z, wrist_roll, wrist_pitch, wrist_yaw, gripper_pos), mathematically padded to 7-DoF with a static lateral Y-axis to match OpenVLA's pre-trained architecture[cite: 3, 2].
- Normalization: Continuous actions were min-max normalized to a
[-1, 1] continuous space and discretized into 256 Llama tokenizer bins[cite: 3].
Training Procedure
Training Hyperparameters
- Training regime:
bf16 mixed precision
- Optimizer:
adamw_torch
- Learning Rate: 2e-4
- Epochs: 5
- Batch Size: 4 (per device)
- Gradient Accumulation Steps: 4
- LoRA Rank (r): 32
- LoRA Alpha: 64
- LoRA Dropout: 0.05
- Target Modules:
q_proj, v_proj, k_proj, o_proj
Speeds, Sizes, Times
- Total Trainable Parameters: 33,554,432
- Total Model Parameters: 7,574,791,616
- Training Time: ~1 hour 57 minutes (3,325 optimization steps)
- Final Training Loss: 0.5316
Technical Specifications
Compute Infrastructure
The model was trained on the Sol Supercomputer (Arizona State University).
Hardware
- GPU: 1x NVIDIA A100 (64GB VRAM)
- CPU: 8 cores
Software
- PyTorch: 2.2.0+cu121
- Transformers: 4.40.1
- PEFT: 0.11.1
- Flash-Attention: 2.8.3
Framework versions
