
Architecture	ACT (Action Chunking with Transformers)
Vision backbone	ResNet18 (ImageNet-pretrained)
Transformer	dim=256, 4 heads, 2 encoder + 1 decoder layers
VAE	latent_dim=16, kl_weight=10.0
Action space	[speed, steering] continuous, normalized to [-1, 1]
Chunk size	20 steps (~1.3s at 15fps)
Parameters	17.1M
Model size	65 MB

Training


Dataset	100 episodes, 12,037 frames (13.4 min)
Training steps	50,000
Batch size	8
Learning rate	1e-4 (AdamW)
Final loss	0.185
Training time	5h 48m on MacBook Pro M1 Pro (MPS)
Epochs	~33
Framework	LeRobot 0.5.1 + PyTorch 2.10

Inputs / Outputs

Inputs:

observation.image: camera frame [3, 480, 640] float32 (ImageNet-normalized)
observation.state: [speed, steering] float32, normalized to [-1, 1]

Output:

action: [speed, steering] float32, normalized to [-1, 1]

Usage

python

from lerobot.policies.act.modeling_act import ACTPolicy
from lerobot.policies.factory import make_pre_post_processors

# Load policy
policy = ACTPolicy.from_pretrained("pbelevich/lego42176_garage_parking_act")
policy.eval()

# Load normalizers
preprocessor, postprocessor = make_pre_post_processors(
    policy.config, pretrained_path="pbelevich/lego42176_garage_parking_act",
)

# Inference
batch = preprocessor({
    "observation.image": image_tensor,   # [1, 3, 480, 640]
    "observation.state": state_tensor,   # [1, 2]
})
action = policy.select_action(batch)
action = postprocessor(action)           # [1, 2] denormalized

See the project repository for the full inference script that connects to the LEGO car via Bluetooth.

Hardware

Car: LEGO Technic 42176 Porsche GT4 e-Performance (BLE control)
Camera: Dahua IP camera, 640x480 via RTSP
Training: MacBook Pro M1 Pro, PyTorch MPS backend