Keep Calm — Communication Risk Analyzer
A privacy-first, on-device model for pre-send communication analysis.
Detects tone, intent, and communication risk in English and Italian text.
Model description
Three independent single-task models sharing a distilbert-base-multilingual-cased backbone:
- Risk model: regression head predicting continuous 0–1 communication risk
- Tone model: multi-label classification across 5 tones
- Intent model: multi-class classification across 4 intents
Tone labels
neutral · frustrated · hostile · sarcastic · positive
Intent labels
constructive · critical · personal · informational
Intended use
Pre-send analysis of workplace text communication. The user writes a message, invokes Keep Calm, sees the analysis, and decides whether to send, revise, or discard.
Not intended as a moderation or censorship tool. The model estimates perception, not objective truth.
Out-of-scope use
- Automated content moderation
- Post-hoc message flagging
- Surveillance or monitoring without consent
- Analyzing messages in domains other than workplace chat
- Languages other than English and Italian
Training data
13,329 annotated examples (English + Italian), workplace chat domain. Sources: YouTube comments, GitHub PRs/issues, LLM-synthesized samples. All labeled by 3+ culturally diverse annotators.
Bias, risks, and limitations
- Direct communication penalty: users from direct-communication cultures (German, Dutch) may receive higher risk scores
- Sarcasm is hard: the model's weakest tone (F1 = 0.515); low-confidence predictions are surfaced
- Single domain: trained only on workplace chat; cross-domain performance unmeasured
- Context-blind: no conversation history, relationship context, or cultural cues
- Subjective ground truth: annotator agreement reflects the inherent subjectivity of communication perception
- Intent classification: the weakest task at 70.6% accuracy
Evaluation results
| Task | Metric | Score |
|---|
| Risk | MAE | 0.100 |
| Risk | Pearson r | 0.700 |
| Risk | Level accuracy | 90.6% |
| Tone | Macro F1 | 0.677 |
| Intent | Accuracy | 70.6% |
Latency: 12.3ms per message on Apple M1 (CPU-only).
Bias audit FP rate: 15.7% (51 curated probes across 9 categories).
Hardware
- Inference: CPU-only, ~400MB RAM, ~12ms per message
- Training: single 16GB GPU (reference: RTX 5060 Ti)
How to use
1from keep_calm import KeepCalmAnalyzer
2
3analyzer = KeepCalmAnalyzer()
4result = analyzer.analyze("Your message here")
5
6print(result.communication_risk) # 0.72
7print(result.risk_level) # RiskLevel.HIGH
8print(result.explanation) # human-readable
Citation
1@software{keep_calm,
2 title = {Keep Calm: Pre-send Communication Risk Analysis},
3 year = {2026},
4 url = {https://github.com/elchief84/keep-calm}
5}
License
Apache 2.0
