Views
No views yet
distilbert-base-multilingual-cased, exported to ONNX (fp32 + INT8) for cheap CPU
serving.0 negative · 1 neutral · 2 positive. signed = P(pos) − P(neg).[neg 1.2, neu 1.5, pos 0.5].| Path | Size | Use |
|---|---|---|
model.onnx | 541 MB | fp32 ONNX (self-contained) |
onnx_int8/model_quantized.onnx | 136 MB | INT8 — the deployed artifact (p50 ~7 ms CPU) |
tokenizer.json, vocab.txt, *config*.json | — | WordPiece tokenizer + config |
1import onnxruntime as ort, numpy as np
2from transformers import AutoTokenizer
3
4tok = AutoTokenizer.from_pretrained("<this-repo>")
5sess = ort.InferenceSession("onnx_int8/model_quantized.onnx", providers=["CPUExecutionProvider"])
6enc = tok(["soul clutch insane", "godlike choked again"], padding=True, truncation=True,
7 max_length=64, return_tensors="np")
8logits = sess.run(None, {k: v for k, v in enc.items() if k in {i.name for i in sess.get_inputs()}})[0]
9p = np.exp(logits) / np.exp(logits).sum(-1, keepdims=True)
10print(p.argmax(-1)) # 0 neg / 1 neu / 2 pos