You can deploy this model and start earning money today!

asgard-robot/
s0101-act-potato-handover

Name: s0101-act-potato-handover
Rating: 2.5 (6 reviews)

Views

No views yet

ACT: Potato Handover (s0101_act_potato_handover)

Action Chunking Transformer model fine-tuned on ASGARD potato handover demonstrations

This is an ACT (Action Chunking Transformer) model trained on 40 episodes of potato manipulation and handover demonstrations from the ASGARD so101_follower robot arm.

Model Details

Model Type

Architecture: ACT (Action Chunking Transformers)
Parameters: ~52M
Task: Potato manipulation and handover
Checkpoint: Step 1800 (final)

Training Configuration

Dataset: asgard-robot/asgard_training_data_potato
Episodes: 40 demonstrations
Total Frames: 30,795
Robot: ASGARD so101_follower (single-arm 6 DOF)
Training Steps: 1,800
Logged Epochs: ~119 (with MetricsTracker bug accounting)

Hardware

GPUs: 4x NVIDIA H100 PCIe (80GB VRAM each)
Total VRAM: ~320GB
Effective Batch Size: 512 (128 per GPU × 4 GPUs)
Training Time: ~17 minutes

Hyperparameters

Learning Rate: 1e-5
Learning Rate (Backbone): 1e-5
Weight Decay: 1e-4
Batch Size: 128 per GPU
Optimizer: AdamW (betas: 0.9, 0.999, eps: 1e-8)
Gradient Clipping: 10.0

ACT-Specific Parameters

Chunk Size: 100
Action Steps: 100
VAE Training: Yes
KL Weight: 10.0
Dropout: 0.1

Architecture Details

Vision Backbone: ResNet-18 (pretrained on ImageNet)
Hidden Dimension: 512
Feedforward Dimension: 3,200
Attention Heads: 8
Encoder Layers: 4
Decoder Layers: 1
VAE Encoder Layers: 4
Latent Dimension: 32

Training Results

Initial Loss: 0.632
Final Loss: 0.245 (at step 1800)
Loss Reduction: 61%
Training Speed: ~1.9 steps/second
Memory Usage: ~40-50GB per GPU

Model Files

model.safetensors: 198MB (model weights)
config.json: ACT configuration
train_config.json: Training configuration
policy_preprocessor.json: Input preprocessing
policy_postprocessor.json: Output postprocessing
Normalizer weights: 7.5KB each

Intended Use

This model is designed for:

Potato manipulation: Picking up, cleaning, and handling potatoes
Handover operations: Coordinated handoff to humans
Fine-grained manipulation: Precise gripper control
Home assistant applications: Domestic robot tasks

Performance Characteristics

Training Metrics

Smooth loss convergence
No overfitting observed
Stable gradient magnitudes (~14-20)
Consistent learning throughout training

Expected Performance

High success rate on similar potato manipulation tasks
Smooth action generation
Proper force control during handover
Robust to slight variations in setup

Limitations

Limited demonstrations: Trained on 40 episodes only
Overfitting risk: May not generalize to drastically different scenarios
Camera dependency: Requires similar camera setup (wrist + external)
Robot-specific: Designed for ASGARD so101_follower robot
Task-specific: Optimized for potato handover task

Usage Example

python

from lerobot.scripts.lerobot_train import load_policy
import torch

# Load the trained model
policy = load_policy(
    "asgard-robot/s0101-act-potato-handover",
    device="cuda"
)

# Run inference
# observation should contain:
# - Images from wrist and external cameras
# - Current joint positions
action = policy(observation)

Dataset

Source: ASGARD teleoperation demonstrations
Format: LeRobot v3.0
Cameras: Dual RGB (wrist + external)
Control: 6 DOF joint positions
Frequency: 30 FPS

Training Environment

Framework: LeRobot (Hugging Face)
Branch: ASGARD teleop control
Python: 3.10
PyTorch: With CUDA support
Multi-GPU: Distributed training with Accelerate

Citation

If you use this model, please cite:

bibtex

@misc{s0101_act_potato_handover_2024,
  title={ACT: Potato Handover for ASGARD Robot},
  author={ASGARD Team},
  year={2024},
  url={https://huggingface.co/asgard-robot/s0101-act-potato-handover}
}

Model Card Author

ASGARD Robot Team

Acknowledgments

Base architecture: ACT (Zhao et al., 2023) - Learning Fine-Grained Bimanual Manipulation
Training framework: LeRobot by Hugging Face
Hardware: Shadeform H100 Multi-GPU Cluster
Dataset: Collected via ASGARD teleoperation system