ModernBERT Intent Router - Quantized ONNX
Model Description
This model is an optimized version of atekrugis/intent-router-v11-modernbert. It uses a ModernBERT transformer body for generating embeddings and an XGBoost forest head for intent classification.
Quantization Details
- Format: ONNX
- Optimization Strategy: Dynamic Int8 Quantization
- Quantization Tool:
onnxruntime.quantization
Results
- Original Size: 568.93 MB
- Quantized Size: 142.96 MB
- Size Reduction: 74.9%
Baseline Performance (Pre-Quantization)
- Math Intent: 99.7%
- Coding Intent: 99.9%
- Weather (Agentic) Intent: 99.4%
Usage
This model is designed for high-efficiency CPU inference using ONNX Runtime with significantly reduced memory footprint.
