GhostAI v2 (distilled)
Second iteration of the Ghost AI on-device planner/executor, fine-tuned from
Qwen3.5-0.8B. Supersedes
ghostai-alpha.
What changed vs alpha
| alpha | v2 |
|---|
| training data | deterministic templates | teacher-distilled (Qwen3.6-35B-A3B, rejection-sampled) |
| train rows | 918 | 2,969 |
| unique plan summaries | 33 / 400 (8%) | ~1,074 / 1,122 (96%) |
| unique clarify questions | 3 / 72 (4%) | 343 / 356 (96%) |
| eval split | 25% of rows had a train twin | 0% — split on utterance fingerprint |
| tools exercised | 25 / 36 | 31 / 36 |
Every plan passed four gates before entering the corpus: scenario contract
(intent / must_call / must_not_call / flags), value-moving completeness, Layer-1
deterministic checks, and Layer-2 simulated execution.
Files
| File | Size |
|---|
ghostai-v2-F16.gguf | 1.4 GB |
ghostai-v2-Q8_0.gguf | 774 MB |
ghostai-v2-Q4_K_M.gguf | 505 MB — mobile target |
Built from the epoch-1 checkpoint, not the final one: eval loss rose every epoch
(0.2162 → 0.2212 → 0.2403) while accuracy plateaued, so epochs 2–3 only overfit.
Metrics (genuinely held-out split)
| epoch | eval_loss | eval_token_acc |
|---|
| 1 (shipped) | 0.2162 | 0.9449 |
| 2 | 0.2212 | 0.9471 |
| 3 | 0.2403 | 0.9472 |
Final train loss 0.1843. Eval loss sits above train loss and eval accuracy below
train accuracy — the correct direction, and the first time these numbers measure
generalization rather than memorization.
Usage
1llama-cli -m ghostai-v2-Q4_K_M.gguf -ngl 99 --temp 0 --jinja \
2 --chat-template-kwargs '{"enable_thinking":false}' \
3 -sys "You are Ghost AI, an on-device mobile crypto assistant. Produce a structured plan JSON matching the v1 schema." \
4 -p "swap 2 SOL to USDC"
Use grammar-constrained decoding in production — the plan schema when planning, the
tool's own JSON schema when executing. Unconstrained output is not reliably schema-valid.
Note: config.json sets mtp_num_hidden_layers: 0. The base checkpoint's Multi-Token-
Prediction head is not carried through, and llama.cpp otherwise expects a 25th block.
Known limitations
- Non-SOL transfers do not resolve the token.
send 50000 BONK to alice still emits
send_sol without a preceding search_token, violating provenance rule 1. The corpus
contains correct examples (70/70 of the SPL-transfer plans call search_token) but
they are outvoted 38-to-130 in the training split, so the model learned the majority
"no lookup" pattern. Being addressed by widening that coverage.
- Unrecognised tokens are not clarified.
send 10 hood to mystic.seeker guesses a
send rather than asking which token is meant. Same cause — only 12 such examples.
- 5 of 36 tools untrained:
add_contact, get_address_balance,
get_transaction_detail, prediction_claim, prediction_sell.
- Weakest categories in the corpus:
planning/private-swap (60%), planning/swap (73%).
Swap completeness is fixed: value-moving swap plans now carry through to
execute_swap instead of stopping at the quote.
License
Base model Qwen/Qwen3.5-0.8B is Apache 2.0; that license governs this derivative.
