Unitree Go2 — ROS 2 Navigation, Mapping and Low-level Control & Basic Vision-Language-Action
A ROS 2 workspace for the Unitree Go2 quadruped that pairs a classic navigation
stack (SLAM Toolbox mapping + Nav2) with a Vision-Language-Action (VLA) layer:
the robot can be driven by typing a natural-language instruction instead of a
joystick or a Nav2 goal. Two interchangeable VLA brains are included — a local
Qwen2.5-VL reasoning loop and NaVILA (VILA-8B), a model purpose-trained
for language-guided navigation.
1cddocker2cp .env.example .env # if you keep one locally; otherwise export ROBOT_IP inline3ROBOT_IP=<go2-lan-ip>docker compose up go2 # driver + Nav2 stack4ROBOT_IP=<go2-lan-ip>docker compose --profile vla up vlm # + local VLA web console (GPU)
The image builds on ros:jazzy-ros-base, matching the distro this project is
developed against. network_mode: host is required for ROS 2 DDS discovery and
for the driver's WebRTC session with the robot on the LAN. The vlm service
requests a GPU via the NVIDIA Container Toolkit —
install that on the host first. NaVILA is not containerized: it needs its
own conda environment, a hand-patched transformers build for Blackwell GPUs,
and a ~16 GB checkpoint pulled at setup time, so it's run on bare metal via
navila/setup_navila_blackwell.sh — see the
NaVILA section below.
Never bake a real ROBOT_IP/ROBOT_TOKEN into the image or into
go2_navigation/config/robot.yaml — pass them as environment variables at run
time instead.
Usage
Put the Go2 in Wi-Fi/STA mode and note its LAN IP (mobile app → Device → Data →
Automatic Machine Inspection → STA Network: wlan0). Set it once in
go2_navigation/config/robot.yaml or pass
ROBOT_IP=<ip> on any launch command (env always wins).
source install/setup.bash
Mapping
ros2 launch go2_navigation mapping.launch.py
Mark a dock rectangle with tape as a fixed starting point. In the SlamToolbox
panel in rviz, select Start At Dock, drive the robot around with a
controller to build the map, then Save Map / Serialize Map.
Autonomous navigation (Nav2)
ros2 launch go2_navigation navigation.launch.py
Loads a saved map and drives the robot with Nav2's standard global/local
planner stack + AMCL localization — set goals from rviz as usual.
Manual teleop
ros2 run go2_control keyboard_teleop
Arrows = move, q/e = rotate, space = stop, x = quit. Releasing all keys
for >0.4 s triggers a dead-man stop.
VLA navigation
Instead of a Nav2 goal, type an instruction and let a vision-language model
decide the motion step by step from the live camera feed. No RL or learned
locomotion is involved anywhere in this stack — every brain below only ever
emits a discrete motion command (move forward 25 cm, turn left 30°,
stop); the Go2's own built-in gait and this repo's closed-loop
MotionController (odom + lidar + E-Stop) execute it.
Open http://localhost:8001, type a goal in natural language (e.g. "go to
the water bottle, turn right, stop at the chair") or a manual command (move forward 75 cm, turn left 90 deg). Each step: camera frame → Qwen2.5-VL
prompt (with live odom/lidar/bbox metrics injected as state) → JSON
{action, value, unit, is_finished, ...} → MotionController executes →
repeat until is_finished or Stop/E-Stop.
Two control modes (VLA_CONTROL):
vlm (default) — the model reasons over injected state and returns the next action.
servo — continuous closed-loop visual servoing driven by YOLO detections,
bypassing the LLM once a target is locked on.
NaVILA is an 8B vision-language-action
model trained specifically for language-guided navigation. It's integrated as a
System-2 / System-1 split brain:
System-2 (navila_server.py, its own conda env, port 8100) loads the
8B checkpoint once and turns 8 camera frames + the instruction into an
action sentence ("The next action is move forward 25 cm.") — a pure
action policy, no chain-of-thought.
System-1 is the same odom/lidar-closed-loop MotionController used
above; NaVILA never touches locomotion or gait, only high-level intent.
First-time setup (downloads the code repo + checkpoint, patches transformers
for Blackwell GPUs):
bash navila/setup_navila_blackwell.sh
One-command run (driver + NaVILA server + web console):
shell
1cd ros2_ws/src/navila
2./scripts/run.sh
Open http://localhost:8001 and type an instruction in English (NaVILA is
trained on English R2R-style instructions), e.g. "Walk forward down the
hallway and stop near the chair." The reasoning panel shows the raw NaVILA
output, per-token confidence, and a link to the exact 8 frames the model
received.
Configuration reference — NaVILA brain
Env / arg
Default
Meaning
VLA_BRAIN=navila
—
routes control to the NaVILA server
VLA_NAVILA_URL
http://127.0.0.1:8100
NaVILA server address
VLA_NAVILA_FRAMES
8
frames sent per request (must match server --num-frames)
VLA_NAVILA_HISTORY_MAX
64
raw history frames kept per instruction; uniformly resampled down to VLA_NAVILA_FRAMES, always including the first and current frame (matches the paper's sampling scheme)
NAVILA_FALLBACK
stop
behavior when the model output fails to parse: stop | scan
NAVILA_PREPROC
pad
frame preprocessing: fov (undistort + forward crop, closest to training distribution) | pad (full fisheye) | resize | crop
server --model-path
—
path to the downloaded checkpoint
server --port
8100
HTTP port
server --num-frames
8
frames processed per request
Full parameter tables, the three Blackwell inference fixes (8-bit + eager
attention + 4D causal mask — see navila/setup_navila_blackwell.sh),
and eval/debug tooling (navila/tools/) are documented inline in that script
and in navila/scripts/.
NaVILA — An-Chieh Cheng et al., "NaVILA: Legged Robot Vision-Language-Action
Model for Navigation", 2024. Code: AnjieCheng/NaVILA,
checkpoint: a8cheng/navila-llama3-8b-8f on Hugging Face. See the official
repository for the paper and up-to-date citation.
Qwen2.5-VL — Alibaba Qwen team, model Qwen/Qwen2.5-VL-3B-Instruct.
Point Cloud Library (PCL) — used for the SOR/ROR filtering pipeline in go2_navigation.
twist_mux — ros-teleop/twist_mux,
used to arbitrate teleop/VLA vs. Nav2 velocity commands.
aiortc — vendored WebRTC implementation used by the Go2 driver.
License
BSD 2-Clause — see LICENSE. Note: some vendored/derived source
files under go2_navigation/ carry a BSD-3-Clause header inherited from
upstream; if you plan to redistribute, reconcile this with project counsel
before publishing rather than assuming one license governs the whole tree.
