Spark Hermes Cost Router — local Spark $0 → OpenRouter cheap → frontier
A deterministic three-tier cost router for the NVIDIA DGX Spark (GB10, 128 GB unified): every prompt routes to the cheapest tier whose deterministic predicate clears, with the local Qwen3-30B-A3B MoE as the always-warm $0 floor and OpenRouter overflow tiers for the prompts local can't reliably answer.
What this harness is
When does local stop being enough? Measure first, then route.
A Spark holds one strong model warm at a time and pays no per-token cost for it. A frontier model on OpenRouter is the per-token-billed ceiling. The interesting decision is when to escalate — and the only honest answer is the measured leak rate, not the public-docs 60-80% cost-savings pitch. This router ships the predicates that decide, plus the snapshot prices that let you reproduce the dollar curve.
Good for:
- Route a Hermes agent prompt to the cheapest tier whose predicate clears (local Spark / OpenRouter cheap / OpenRouter frontier).
- Embed a deterministic, auditable cost router into a Hermes config (no LLM-classifier overhead).
- Reproduce the H6 leak-rate measurement on a custom workload by re-grading the same 12-prompt shape.
For: DGX Spark power users running a local-first agent harness who want to escalate to frontier only when local can't reliably answer — and to know what that fraction is.
Serving lanes
| Lane | Provider | Model | tok/s | Sustained (min) | $/M input | $/M output |
|---|
| Local Spark — Qwen3-30B-A3B MoE Q4_K_M | llama-server | Qwen3-30B-A3B-Q4_K_M.gguf | — | — | $0.00/M | $0.00/M |
| OpenRouter cheap-tier — gpt-4o-mini | openrouter | openai/gpt-4o-mini | — | — | $0.15/M | $0.60/M |
| OpenRouter frontier — claude-opus-4.1 | openrouter | anthropic/claude-opus-4.1 | — | — | $15.00/M | $75.00/M |
The dollar columns are the per-tier price snapshot at measurement time. The cost-routed strategy's per-task spend is a weighted average across tiers — see the article for the measured curve.
Configuration
~/.hermes/config.yaml (model block):
1model:
2 provider: custom
3 base_url: "http://127.0.0.1:8080/v1"
4 default: Qwen3-30B-A3B-Q4_K_M.gguf
~/.hermes/.env:
1HERMES_STREAM_READ_TIMEOUT=1800
2OPENAI_API_KEY=local
3OPENAI_BASE_URL=http://127.0.0.1:8080/v1
4OPENROUTER_API_KEY=<your-openrouter-key>
router.yaml:
1router:
2 kind: cost
3 tiers:
4 - name: simple
5 endpoint: "http://127.0.0.1:8080/v1"
6 model: Qwen3-30B-A3B-Q4_K_M.gguf
7 notes: "Local Spark lane via llama.cpp on :8080; the Step-2 pinned brain. $0 marginal."
8 - name: standard
9 endpoint: "https://openrouter.ai/api/v1"
10 model: openai/gpt-4o-mini
11 complexity_keywords:
12 - summarize
13 - compare
14 - analyze
15 min_input_tokens: 600
16 price_per_m_input_usd: 0.15
17 price_per_m_output_usd: 0.6
18 api_key_env: OPENROUTER_API_KEY
19 notes: OpenRouter cheap-tier value pick (128K ctx).
20 - name: complex
21 endpoint: "https://openrouter.ai/api/v1"
22 model: anthropic/claude-opus-4.1
23 complexity_keywords:
24 - prove
25 - derive
26 - multi-step
27 - step through
28 min_input_tokens: 3000
29 price_per_m_input_usd: 15.0
30 price_per_m_output_usd: 75.0
31 api_key_env: OPENROUTER_API_KEY
32 notes: OpenRouter frontier-tier hard-reasoning pick (200K ctx).
Doctor checklist
Methods
Known drift
- Suite size — 12 prompts × N=3 attempts per strategy (108 calls per full run). Not a large-N guarantee; production workloads will exhibit their own leak rates.
- OpenRouter snapshot prices — Captured 2026-05-28T14:32:06.836115+00:00. openai/gpt-4o-mini = $0.15/M input + $0.6/M output; anthropic/claude-opus-4.1 = $15.0/M input + $75.0/M output. Prices change; re-snapshot before reproduction.
- Leak rate — 33.3% measured leak rate. Tuned to this 12-prompt suite's synthetic-but-graded difficulty distribution.
- Token threshold — complex-tier
min_input_tokens=3000 was tuned to this suite. A workload with a different long-to-short ratio should re-tune this single integer.
Published by
Orionfold LLC ·
orionfold.com · Methods documented at
ainative.business/field-notes.
