Views
No views yet
device → agent. The device noticed, the agent thought, the device acted -- but only if you wired the response path yourself. v1.2 builds the full return path:IMU + Camera + Microphone
↓
TriggerDetector + AudioTriggerDetector
↓
TriggerArbiter (FIRST_WINS / AUDIO_PRIORITY / HIGHEST_CONFIDENCE)
↓
TransportHal (BLE · HTTP · LocalMLX · Attachment)
↓
OpenClaw Agent Runtime
↓
AgentResponseListener
↓
AudioOutputHal.speak() + DisplayHal.show_card() + ActuatorHal.execute()core/response.py). The agent's response now flows back to the device automatically. TEXT routes to AudioOutputHal (TTS) with DisplayHal fallback. DISPLAY routes to the face display. ACTION fires the actuator. HEARTBEAT is acknowledged. No polling; event-driven.transport/mlx.py). On-device inference via mlx_lm on Apple Silicon. Default model: mlx-community/Qwen3-0.6B-4bit. Load time: 8.6s. Inference: 0.59s. Zero cloud dependency. Hybrid routing: local fast path, HTTP gateway fallback for complex reasoning. Install: pip install mlx-lm.triggers/audio_trigger.py). RMS energy threshold detection using arecord (ALSA). State machine: IDLE → DETECTING → TRIGGERED → COOLDOWN. Configurable threshold, min duration, cooldown. Runs concurrently with visual TriggerDetector.triggers/arbiter.py). Fuses visual + audio trigger streams. Policies: FIRST_WINS (default, lowest latency), AUDIO_PRIORITY, VISUAL_PRIORITY, HIGHEST_CONFIDENCE (holds window, emits best confidence). 500ms deduplication window prevents double-firing.set_color(r, g, b), blink(interval_ms), pulse(pattern), off(). Patterns: heartbeat, alert, processing, idle. Simulator included for CI.get_expected_latency_ms() is now abstract -- every transport declares its expected overhead. BLE: 50ms. HTTP: 10ms. LocalMLX: 5ms. AttachmentTransport: 100ms. Pipeline uses this to offset actuation timing for synchronized TTS + display delivery.transport/attachment.py). Attaches raw camera frames (base64) directly to openclaw sessions spawn turns. The agent sees the actual frame, not a compressed embedding. Useful for high-value visual context where BLE's 25KB limit isn't enough.openclaw stt transcribe via subprocess. All device profiles get STT without a per-device speech stack. Async variant included.TriggerDetector calls POST /heartbeat/trigger on the OpenClaw gateway at every CAPTURE transition. Eliminates up to 30s agent polling latency. 5s cooldown prevents spam.hal/reachy2_reference.py). Full humanoid: 7-DOF arms × 2, grippers, 3-DOF neck, stereo cameras, optional wheeled mobile base. Same HAL ABCs as Reachy Mini, richer actuator map. load_profile("reachy2", host="reachy.local").| Metric | v1.1 (Gate 3) | v1.2 (Gate 4) |
|---|---|---|
| HAL ABCs | 8 | 9 |
| Device profiles | 6 | 7 |
| Tests passing | 237 | 270 |
| Bidirectional loop | No | ✅ Yes |
| Offline inference | No | ✅ Yes |
| Multi-modal triggers | No | ✅ Yes |
pip install openclaw-embodiment && openclaw-embodiment doctordocs/specs/):HalOrchestrator: explicit trigger → capture → classify → transport → actuate loop as first-class async object with per-stage hooks and middlewareCrossEmbodimentOrchestrator: one agent, multiple simultaneous device types (glasses + robot + phone) with intent routing