Views
No views yet
| Feature | Description |
|---|---|
| 🧠 Hierarchical Planner | Breaks goals into subtasks before execution using a cheap text model |
| 🔌 Playwright MCP | Semantic browser control (click by text/role, extract tables/links, evaluate JS) |
| 🎯 Multi-Model Router | Auto-selects the cheapest capable model (fast vision ↔ powerful vision ↔ fast text ↔ powerful text) |
| 🧩 Set-of-Marks Vision | Overlays numbered bounding boxes on UI elements for coordinate-free interaction |
| 🗄️ Long-Term Memory | ChromaDB vector store retrieves similar past tasks and proven strategies |
| 🔍 Verifier Agent | Checks subtask completion and triggers recovery loops |
| 🛑 Human-in-the-Loop | Pauses on sensitive actions (payments, emails, deletes) for user approval |
| 🎙️ Voice I/O | Speak tasks and hear responses via Whisper STT + Kokoro TTS |
| 💰 Cost Dashboard | Real-time $/task, token usage, and latency tracking |
| 📹 Session Recording | Saves every step as replayable macros with GIF/MP4 export potential |
| 🧪 Enhanced Eval | Built-in benchmark suite with LLM-as-a-Judge grading and A/B testing |
User Input (Text / Voice / File)
|
v
[Intelligence Router] ----> Planner (JSON DAG)
|
v
[Memory Retrieval] (ChromaDB)
|
v
[Plan Executor]
|
+---> [Browser Sub-Agent] (Playwright MCP)
+---> [Desktop Sub-Agent] (E2B + SoM Vision)
+---> [Coder Sub-Agent] (Code Interpreter)
+---> [HF Hub Sub-Agent] (Search / Upload)
|
v
[Verifier] -> Retry / Alternative / Continue
|
v
[Macro Saver] + Cost Report + Session Recording1from eval_harness import EvaluationHarness
2# See eval_harness.py for usage