Repo: learn-abc/banking77-intent-classifier-en
Banking77 Intent Classifier (12-Intent)
Overview
This model is a fine-tuned BERT-based intent classifier designed for banking and financial customer queries.
It is trained by mapping the original 77 Banking77 intents into a smaller, production-friendly set of custom intents, making it suitable for real-world conversational systems where simpler intent routing is required.
The model performs single-label text classification and is intended to be used as an intent detection component, not as a conversational or generative model.
Model Details
- Base model:
bert-base-uncased
- Task: Text Classification (Intent Classification)
- Architecture:
BertForSequenceClassification
- Languages: English (robust to informal and conversational phrasing)
- Max sequence length: 64 tokens
- Output: One intent label with confidence score
Custom Intent Schema
The original 77 Banking77 intents were mapped and consolidated into the following 12 production intents:
ACCOUNT_INFOATM_SUPPORTCARD_ISSUECARD_MANAGEMENTCARD_REPLACEMENTCHECK_BALANCEEDIT_PERSONAL_DETAILSFAILED_TRANSFERFEESLOST_OR_STOLEN_CARDMINI_STATEMENTTRANSFERFALLBACKGREETING
Any user query that does not clearly belong to one of the supported categories is mapped to FALLBACK.
This design simplifies downstream business logic while retaining strong intent separation.
Training Data
-
Primary dataset: PolyAI Banking77
-
Original training samples: 10,003
-
Test samples: 3,080
-
After intent mapping and augmentation:
- Training samples: 22,256
- Includes: 570 explicitly added
FALLBACK examples
Training Intent Distribution (Post-Mapping)
| Intent | Samples |
|---|
| MINI_STATEMENT | 4090 |
| ACCOUNT_INFO | 3843 |
| FEES | 2918 |
| FAILED_TRANSFER | 2050 |
| CARD_MANAGEMENT | 2005 |
| ATM_SUPPORT | 1459 |
| CARD_REPLACEMENT | 1456 |
| CHECK_BALANCE | 1092 |
| CARD_ISSUE | 902 |
| TRANSFER | 673 |
| FALLBACK | 570 |
| GREETING | 520 |
| LOST_OR_STOLEN_CARD | 445 |
| EDIT_PERSONAL_DETAILS | 233 |
Class imbalance was handled using class weighting during training.
Evaluation Results
Final evaluation on the Banking77 test set:
- Accuracy: 95.7%
- F1 (Micro): 0.960
- F1 (Macro): 0.956
These results indicate strong overall performance with good balance across both high-frequency and low-frequency intents.
Usage
Load the model
1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4model_id = "learn-abc/banking77-intent-classifier-en"
5
6tokenizer = AutoTokenizer.from_pretrained(model_id)
7model = AutoModelForSequenceClassification.from_pretrained(model_id)
8
9def predict_intent(text):
10 inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
11 with torch.no_grad():
12 outputs = model(**inputs)
13 probs = torch.softmax(outputs.logits, dim=-1)
14 pred_id = probs.argmax(dim=-1).item()
15 confidence = probs[0][pred_id].item()
16
17 return model.config.id2label[pred_id], confidence
18
19# Example usage:
20if __name__ == "__main__":
21 test_texts = [
22 "What is my account balance?",
23 "Show me my last 10 transactions.",
24 "I want to update my address.",
25 "How do I apply for a loan?"
26 ]
27
28 for text in test_texts:
29 intent, confidence = predict_intent(text)
30 print(f"Input: {text}\nPredicted Intent: {intent} (Confidence: {confidence:.2f})\n")
Intended Use
This model is suitable for:
- Banking chatbots
- Voice assistant intent routing
- Customer support automation
- FAQ classification systems
It is designed to be used together with business rules, confirmation flows, and fallback handling.
Limitations and Safety Notes
- The model does not perform authentication or authorization
- It must not directly trigger financial actions
- High-risk intents (e.g. lost or stolen card) should always require explicit user confirmation
- Predictions should be validated with confidence thresholds and fallback logic
This model is not a replacement for human review in sensitive workflows.
Notes on Model Warnings
During training, warnings related to missing or unexpected keys were observed.
These are expected when fine-tuning a pre-trained BERT checkpoint for a downstream classification task and do not impact inference correctness.
Citation
If you use this model, please cite:
- Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- PolyAI Banking77 Dataset
Maintainer
Developed and fine-tuned for production-oriented banking intent classification.
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors
