model_int8.onnx — INT8 quantized ONNX modelvocab.txt — WordPiece vocabularylabels.txt — NER label set (line N = label ID N)1import onnxruntime as ort
2import numpy as np
3
4session = ort.InferenceSession("model_int8.onnx")
5inputs = {
6 "input_ids": np.array([[...]], dtype=np.int64),
7 "attention_mask": np.array([[...]], dtype=np.int64),
8 "token_type_ids": np.array([[...]], dtype=np.int64),
9}
10logits = session.run(None, inputs)[0] # (batch, seq_len, num_labels)