Fine-tuned
X-VLA checkpoints on the
BEHAVIOR-1K 2025 challenge demonstrations, with
weighted cross-entropy loss for the skill classifier.
The original skill classifier suffered from severe class imbalance — "move to" accounts for 34% of all frames, causing the classifier to always predict that single skill. This release fixes the issue with sqrt-inverse frequency class weighting:
This ensures the model learns to distinguish all 34 skill primitives, not just the dominant one.
Training is ongoing — 200k checkpoints and additional LR variants (1e-4) will be uploaded as they complete.
-
Additive Task + Skill Soft Prompts: Separate task_prompt_hub and skill_prompt_hub embeddings replace the single domain prompt. Combined additively: prompt = task_prompt[task_id] + skill_prompt[skill_id].
-
VLM-based Skill Classifier: A linear head on pooled VLM features predicts the current skill primitive (34 classes). Trained with auxiliary weighted CE loss (λ=0.1). At inference, auto-predicts skill when ground truth is unavailable.
-
Enriched Language Instructions: Training appends skill + object context to task descriptions, e.g., "Turn on the radio. Current: pick up radio from coffee table."
-
23D Action Space: base_qvel(3) + trunk_qpos(4) + arm_left(7) + grip_left(1) + arm_right(7) + grip_right(1) with delta-joint transformation on 17 dims.
1from models.modeling_xvla import XVLA
2from models.processing_xvla import XVLAProcessor
3
4model = XVLA.from_pretrained("Hoshipu/xvla-behavior1k-weighted-ce/single_task_0_100k")
5processor = XVLAProcessor.from_pretrained("Hoshipu/xvla-behavior1k-weighted-ce/single_task_0_100k")
