Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning
Authors: Yuya Shimizu¹, Yongdong Wang^{1,2}, So Shimooka¹, Tetsushi Kamegawa¹
¹ Graduate School of Environmental, Life, Natural Science and Technology, Okayama University
² Department of Precision Engineering, The University of Tokyo
Project Page: https://yuya-0411.github.io/project-crawl/
Year: 2026
This work was supported by OU-SPRING, JSPS KAKENHI Grant Number 26K07423, and JSPS KAKENHI Grant Number 26K21350.
Abstract
Traditional gait design for snake robots has long relied on geometric models inspired by biological snakes. However, due to the inherent morphological discrepancies between biological snakes and robotic systems, these conventional models are not always optimal for snake robots.
In this study, we trained a 10-DoF snake robot on random step fields using reinforcement learning. The learned policy autonomously emerged dynamic gaits capable of traversing unstructured terrains where conventional sine-wave-based locomotion typically fails. The robot effectively utilized environmental protrusions as pivot points for propulsion, three-dimensionally transforming its body to overcome obstacles.
Although these gaits deviate from predefined periodic motions, they exhibit characteristics approximating biological sidewinding as a direct result of physical constraints. Comparative experiments with geometric models confirm that the learned policy offers superior robustness and adaptability in rugged terrains.
Robot Model
- Structure: 11 links connected by 10 alternating yaw/pitch joints (orthogonally arranged)
- Length: 9 central units × 0.16 m + 2 terminal units × 0.08 m = 1.6 m total
- Sensors: Each link is equipped with a tactile sensor for contact-aware state feedback
Method
Observation & Action Space
The policy takes a 56-dimensional observation as input and outputs 10 joint position targets.
| Category | Item | Dimension |
|---|
| Observation | Joint positions | 10 |
| Joint velocities | 10 |
| Per-link contact forces (11 links × 3 axes, in local frame) | 33 |
| Global orientation (roll, pitch, yaw) | 3 |
| Total | 56 |
| Action | Joint position targets | 10 |
Per-link contact forces are the key embodiment-sensing component — each link's resultant contact force is expressed in its local coordinate frame, capturing normal and tangential friction. Joint targets are scaled as: q_target = (π/2) × a, where a ∈ [−1, 1].
Reward Function
The total reward combines six terms:
| Term | Weight | Purpose |
|---|
| Forward velocity | 5.0 × exp(−(0.5 − v_y)² / 0.25) | Encourage target speed of 0.5 m/s |
| Lateral deviation | −0.5 × |x_com| | Penalize drifting sideways |
| Trajectory correction | +1.0 × max(0, −x_com × v_x) | Self-correct when off-track |
| Ground contact | +0.2 × N_contact | Reward multi-link ground contact |
| Head height | −10.0 if z_head > 0.2 m | Enforce low center-of-gravity posture |
| Action smoothness | −0.005 × Σ(Δa_i)² | Penalize large joint movements |
Training Setup
| Parameter | Value |
|---|
| Algorithm | PPO (RSL-RL) |
| Simulator | NVIDIA Isaac Lab (GPU-accelerated PhysX, 50 Hz) |
| Parallel environments | 4,000 (10 × 20 terrain field, 20 agents/area) |
| Network | MLP [256, 256], ELU activation |
| Learning rate | 2.5 × 10⁻⁵ |
| Discount factor (γ) | 0.99 |
| GAE lambda (λ) | 0.95 |
Training Terrains (4 types)
| Type | Description | Parameter Range |
|---|
| Pyramidal Stairs & Pits | Multi-level elevation changes | Step height 0.05–0.23 m |
| Random Grid (Boxes) | Discontinuous rigid obstacles | Grid height 0.05–0.20 m |
| Random Rough Surface | Stochastic height-field noise | Noise range 0.02–0.10 m |
| Pyramidal Slopes | Continuous inclined surfaces | Slope 0.0–0.4 rad |
Checkpoint
model_3000.pt — PPO policy weights at 3,000 training iterations.
1import torch
2
3checkpoint = torch.load("model_3000.pt", map_location="cpu")
4print(checkpoint.keys())
For full simulation and evaluation setup, see the
project page.
Key Findings
- The policy emerged a gait approximating biological sidewinding purely from physical constraints — no gait template was provided.
- The robot exploits terrain protrusions as pivot points for propulsion, reconfiguring its body in 3D to overcome obstacles.
- Against the geometric sidewinding baseline (amplitude 1.2 rad, frequency 0.50 Hz), the learned policy shows superior robustness on unstructured terrain, avoiding the "stacking phenomenon" caused by the open-loop geometric model.
- On flat ground, the learned gait achieves reliable forward locomotion driven by the full 56-D observation, with independent per-joint phase control rather than a prescribed waveform.
Citation
1@inproceedings{shimizu2026emergent,
2 title={Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning},
3 author={Shimizu, Yuya and Wang, Yongdong and Shimooka, So and Kamegawa, Tetsushi},
4 year={2026}
5}
