Therapy Safety Router v3 (LoRA)
Safety layer for wellbeing chatbots. Classifies an incoming user message; never
replies to the user, never diagnoses.
message -> {risk, request, injection_attempt} -> the host system picks an action.
Decide from the probability, not the generated token
Prefill {"risk":" and read the distribution over the next token, then compare
against a threshold. Rounding at 0.5 hides real signal: five of eight rows this
model appeared "blind" to were in fact seen, at p between 0.20 and 0.56.
| threshold | recall (49 risky, held-out) | 95% CI | false alarms |
|---|
| 0.50 | 0.86 | [0.73, 0.93] | 6% |
| 0.40 | 0.92 | [0.81, 0.97] | 6% |
| 0.35 (default) | 0.96 | [0.86, 0.99] | 9% |
| 0.20 | 0.98 | [0.89, 1.00] | 14% |
The threshold is a runtime dial, not a model constant.
Training
Qwen2.5-1.5B-Instruct + LoRA r=16, alpha=32, all linear modules, lr=2e-4,
3 epochs, loss on assistant response only. 526 synthetic rows, labelled by a
3-model jury with planted control rows.
Limits
Not a medical device - one layer among several, human escalation mandatory.
100% synthetic data, English only. The crisis rubric has not been reviewed
by a clinician. Proving recall 0.98 would need ~150 risky eval rows; this eval
has 49.
