Retail Bank Conversation Router
History-aware DistilBERT cross-encoder with one shared encoder and seven
jointly trained heads: domain, lane, family, intent, relation, action, and
entity resolution. Runtime hierarchy constraints expose route, intent,
action, and entity decisions used by the release evaluation.
Held-out results
- Release eligible:
True
- Domain macro F1:
0.995660
- Lane macro F1:
0.990874
- Family macro F1:
0.993572
- Intent macro F1:
0.997157
- Relation macro F1:
0.979800
- Action macro F1:
0.992846
- Entity-resolution macro F1:
0.981114
- Hierarchy compatibility error rate:
0.000000
- Counterfactual action accuracy:
1.000000
- Counterfactual entity accuracy:
1.000000
- Counterfactual exact pair-flip accuracy:
1.000000
- In-domain false-refusal rate:
0.000000
- OOD false-accept rate:
0.006104
- Contextual false-refusal rate:
0.000000
- Repair false-refusal rate:
0.000000
- Topic-shift OOD false-accept rate:
0.000000
Data
The governed corpus contains
20439 train,
4158 validation, and
4921 test rows. Inputs include
only prior visible user/assistant text, pre-turn dialogue state, and the current
user turn; current-turn tool plans, tool results, expected outputs, and final
answers are excluded.
