Every frontier model runs on someone else's cloud. Every agent platform phones home. Every fine-tune assumes NVIDIA.
HERMYTHOS breaks all three assumptions.
Sovereign inference — runs CPU-only on a laptop (5 tok/s on Intel Core Ultra, Q4_K_M). No GPU required. No API key needed.
Open weights — 1.58-bit ternary model derived from Qwen3-8B via QAT on H100, quantized to Q4_K_M (4.7 GB). You own every parameter.
Agent-native engine — 22 Rust crates (29 total), cybernetic loop with 8 tools, 3 frontends (TUI / Flutter Web / Open WebUI). Zero Python at runtime.
The model itself is only half the equation — the Rust engine (hermythos-server) provides the agentic scaffolding: tool execution, FSM-based cybernetic loops, memory persistence, and multi-agent LLM debate via RecursiveMAS.
Quick Install
bash
1# One command. Downloads the model + engine.2curl -fsSL https://raw.githubusercontent.com/drissman/hermythos/main/scripts/install.sh |bash
Or manually:
bash
1# 1. Get the model2hf download Drissman/hermythos-rdt bonsai-rdt-q4_k_m.gguf --local-dir ./models
34# 2. Clone the engine5git clone https://github.com/drissman/hermythos
6cd hermythos-rdt
7cargo run -p hermythos-server --release -- --model ./models/bonsai-rdt-q4_k_m.gguf
Standard LLMs use 16-bit floats per weight. That's 16 GB for an 8B model. Ternary packs weights into {-1, 0, +1} — 12-16× denser — and eliminates multiplication from inference entirely.
Runs on CPU — no GPU required. Laptop-grade Core Ultra gets 5 tok/s.
Runs on RISC-V — no CUDA dependency. No NVIDIA lock-in.
12× smaller — 4.7 GB fits in RAM + disk of any machine built after 2015.
The trade-off is training complexity: ternary quantization requires QAT (Quantization-Aware Training) with Straight-Through Estimator. This model was fine-tuned on an H100 — but inference runs anywhere.
