from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("khaze0911/banking77-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("khaze0911/banking77-distilbert")

inputs = tokenizer("How do I freeze my card?", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

predicted_class = model.config.id2label[logits.argmax().item()]
print(predicted_class)  # → "freeze_account" or similar

Training

Base model: distilbert-base-uncased
Epochs: 8
Batch size: 32
Learning rate: 3e-5
Max sequence length: 128
Optimizer: AdamW with linear warmup (5% warmup ratio)
Mixed precision: fp16

Limitations

The biggest confusion pairs in the test set (why_verify_identity ↔ verify_my_identity, pending_transfer ↔ transfer_not_received_by_recipient) appear to be labeling ambiguities in the source dataset rather than model failures.