Production-ready vertical intent classifier for LLM chatbot guardrails. Classifies user messages as allow, deny, or abstain to keep financial services chatbots on-topic and secure.
Enterprise chatbots in regulated industries face a critical challenge: users inevitably ask off-topic questions (sports, entertainment, relationship advice) that the underlying LLM will happily answer — exposing the organization to compliance risk, brand damage, and potential liability.
Traditional keyword filters miss nuanced off-topic queries, while LLM-based guardrails are too slow and expensive for real-time inference.
The Solution
IntentGuard uses a tiny, purpose-trained DeBERTa-v3-xsmall model (22M parameters, 2.5MB quantized) to classify user intent in <30ms on CPU. The three-way classification (allow/deny/abstain) enables precise control:
Allow — On-topic for the vertical, pass to the LLM
Deny — Clearly off-topic, block with a polite redirect
Abstain — Ambiguous, escalate to secondary classifier or human review
1import onnxruntime as ort
2from transformers import AutoTokenizer
3import numpy as np
45# Load model and tokenizer6tokenizer = AutoTokenizer.from_pretrained("perfecXion/intentguard-finance")7session = ort.InferenceSession("model.onnx")89# Classify a user message10text ="What are the current mortgage rates for a 30-year fixed loan?"11inputs = tokenizer(text, return_tensors="np", max_length=128, truncation=True, padding="max_length")1213logits = session.run(None,{14"input_ids": inputs["input_ids"],15"attention_mask": inputs["attention_mask"]16})[0]1718labels =["allow","deny","abstain"]19prediction = labels[np.argmax(logits)]20confidence =float(np.max(np.exp(logits)/ np.sum(np.exp(logits))))2122print(f"Intent: {prediction} (confidence: {confidence:.3f})")23# Output: Intent: allow (confidence: 0.998)
Docker
bash
1# Pull and run the container2docker pull ghcr.io/perfecxion/intentguard:finance-1.0
3docker run -p 8080:8080 ghcr.io/perfecxion/intentguard:finance-1.0
45# Classify a message6curl -X POST http://localhost:8080/v1/classify \7 -H "Content-Type: application/json"\8 -d '{"messages": [{"role": "user", "content": "What are the current mortgage rates?"}]}'910# Response: {"intent": "allow", "confidence": 0.998}
pip
bash
1pip install intentguard
23# Python usage4from intentguard import IntentGuard
56guard = IntentGuard.load("finance")7result = guard.classify("What are the current mortgage rates?")8print(result)# Intent(label='allow', confidence=0.998)
