quantized-distilbert-banking77 – AI Model by lewtun | AlphaNeural AI

You can deploy this model and start earning money today!

lewtun/
quantized-distilbert-banking77

like

0

text-classification

autotrain_compatible

endpoints_compatible

Views

No views yet

Quantized-distilbert-banking77

This model is a dynamically quantized version of optimum/distilbert-base-uncased-finetuned-banking77 on the banking77 dataset.

The model was created using the dynamic-quantization notebook from a workshop presented at MLOps World 2022.

It achieves the following results on the evaluation set:

Accuracy

Vanilla model: 92.5%
Quantized model: 92.44%

The quantized model achieves 99.93% accuracy of the FP32 model

Latency

Payload sequence length: 128
Instance type: AWS c6i.xlarge

latency	vanilla transformers	quantized optimum model	improvement
p95	63.24ms	37.06ms	1.71x
avg	62.87ms	37.93ms	1.66x

How to use

python

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained("lewtun/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("lewtun/quantized-distilbert-banking77")

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
classifier("What is the exchange rate like on this app?")