Cipher Air is a LoRA fine-tune of Qwen/Qwen2.5-0.5B-Instruct, trained on every LLM-backed feature of a local-first email assistant: email triage (importance/summary/category JSON), chat, daily-summary synthesis, draft reply, compose assist, and voice-command intent parsing — not just prompted for these tasks, actually trained on them.

Update: added a 6th task, voice-command intent parsing (interpreting a dictated voice command + a candidate address list into a structured action). The original 5-task version had never seen this task and would confidently hallucinate a plausible-looking but entirely fake person/address when it couldn't resolve one — this retrain fixes that; see Training below and generate_voice_intent.py for the fix in detail.

Second update: a security-relevant bug was found and fixed — the previous voice-intent retrain, done through several rounds of incremental continue-training, left this model occasionally complying with prompt-injection attempts embedded in email content on the draft-reply/chat tasks (echoing a fake "security verification" request for banking details back into a drafted reply). Rather than patch that narrowly, this model was retrained fresh from the base in one pass on a properly-rebalanced dataset with much richer injection coverage across every task, not just triage. Verified through repeated adversarial testing against the actual calling application's code path (not just raw single-shot completions): no compliance with any credential/wire-transfer/data-exfiltration injection attempt across multiple test rounds. See Training below.

It's the middle of the three Cipher tiers (cipher-nano / cipher-air / cipher-pro) — a balanced default at 40% of cipher-pro's disk size and 3x the throughput. Cipher is the local-model engine for an unreleased larger email-assistant project — that project isn't public yet, but these weights, the training code, the eval script, and all five dataset generators are fully open now, in this repo.

Why this exists

Most email triage today means sending your inbox to a third-party API. Cipher runs entirely on your own hardware via Ollama — nothing about your email ever leaves your machine.

What's in this repo

cipher-air.Q4_K_M.gguf — the model weights, ready for Ollama
Modelfile — the exact Ollama Modelfile (system prompt + inference params) used in training/eval
train_cipher_air.py / export_gguf_cipher_air.py — the exact scripts used to produce this model (Unsloth LoRA on the base model above)
generate2.py, generate_chat.py, generate_daily_summary.py, generate_draft_reply.py, generate_compose.py, generate_voice_intent.py — the six task-specific synthetic-data generators (produces the full multi-task training set)
eval_voice_intent.py — regression harness for the voice-intent task, including the exact hallucinated-address bug case as a required fixture
eval_triage.py / eval_fixtures.json — a standalone benchmark harness (no external dependencies beyond httpx/pydantic) reproducing the numbers below

Everything needed to reproduce this model from scratch, or fine-tune your own variant, is in this repo — nothing here depends on an unreleased package.

Benchmark

Evaluated on a 29-fixture triage benchmark on an RTX 5070:

Model	Disk	Tok/s	JSON-valid	Category acc	Importance-in-band	Injection-safe
cipher-air	398 MB	507.8	100.0%	69.0%	79.3%	100%

Honest caveat: cipher-air is the tightest-capacity tier of the three (only 8.8M of 502M params are trainable via LoRA), and it shows — of the three tiers it's the one most likely to occasionally misjudge whether something genuinely needs a reminder/action versus being routine. cipher-pro and cipher-nano both handle that nuance more reliably. Reproduce with:

bash

pip install -r requirements.txt
python eval_triage.py --models cipher-air:latest --keep

Usage (Ollama)

ollama create cipher-air -f Modelfile

Query it with grammar-constrained JSON output for reliable parsing:

bash

curl http://localhost:11434/api/chat -d '{
  "model": "cipher-air",
  "messages": [
    {"role": "system", "content": "<system prompt from Modelfile>"},
    {"role": "user", "content": "From: alex@acme.com\nSubject: Q3 budget review\n\nBody:\nCan we sync before Friday?"}
  ],
  "format": "json",
  "options": {"temperature": 0.1}
}'

Training

Base: Qwen/Qwen2.5-0.5B-Instruct, LoRA (r=16, alpha=32, all linear layers), 2 epochs
Data: ~4,800 triage examples (oversampled ~2x to ~60% of the final training mix — this size tier needed a stronger triage signal than the other two to hold onto exact JSON schema output while also learning four other task formats) + ~1,600-2,000 examples each for chat/daily-summary/draft-reply/compose, all matching production prompts exactly — generated by the five generate_*.py scripts in this repo
Framework: Unsloth + trl.SFTTrainer
Sequence packing was tried to speed up training (most examples are well under the 2048-token context window) — it crashed outright, an Unsloth/trl version incompatibility, not a quality tradeoff. Disabled.
Reproduce with train_cipher_air.py → export_gguf_cipher_air.py
Voice-intent retrain: added ~1,800 examples from generate_voice_intent.py, weighted heavily toward the no-match case (a spoken name with no corresponding candidate address — the model must return null rather than inventing one) and matching-with-distractors cases. On eval_voice_intent.py's 5 fixtures, cipher-air went from 2/5 (including a fabricated address and an out-of-schema action) to 5/5, the cleanest result of the three tiers on this task.
Full retrain (current version): rather than continue-training the voice-intent adapter further, this version is a fresh LoRA fine-tune from the base model on one consolidated, properly-balanced dataset covering all 6 tasks in a single pass — triage (~4,800, oversampled 3x to hold the ~60% mix ratio this tier needs), chat/daily-summary/draft- reply/compose/voice-intent (~1,800-2,500 each). generate_draft_reply.py and generate_chat.py both gained substantially heavier and more varied injection coverage (credential/wire-transfer/data-exfiltration attempts, not just one generic case) after live testing found the previous incremental-patch version could be induced into complying with an injected "security verification" request for banking details. generate_chat.py also gained an explicit "the question is about something with no connection to your email at all (weather, sports, etc.)" scenario category after finding a hallucination regression there. One clean training pass over the properly-balanced result, instead of a chain of narrow continue-trains, avoids the whack-a-mole pattern where each targeted fix risked nudging a different, previously-working case.

A dead end worth knowing about

We tried quantizing this model down further (Q3_K_M, Q2_K) hoping to shrink it toward cipher-nano's size class. It barely helped (355MB / 339MB vs 398MB at Q4_K_M) — Qwen2.5's 151,936-token vocabulary embedding table dominates disk size and doesn't compress with weight quantization. If you're looking for something genuinely small, use cipher-nano instead (different base model, built specifically to solve this).

License

Apache 2.0, inherited from the base model. Weights, training code, and eval harness are fully open.

srock44/cipher-air

Cipher Air