The historical split contains duplicate/cross-split overlap risk and is retained only as historical evidence. The perfect focused score applies only to the deterministic project stress set. ECE shows that probability calibration is not perfect even though every predicted label was correct on that set.

Machine-readable focused metrics are available in evaluation/minilm_intent_results.json.

Usage

python

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoConfig, AutoTokenizer

repo_id = "Fatihaybasn/pathfinder-minilm-intent-onnx-int8"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
config = AutoConfig.from_pretrained(repo_id)
model_path = hf_hub_download(repo_id, "intent-minilm-int8.onnx")
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])

encoded = tokenizer(
    ["please open the camera"],
    padding=True,
    truncation=True,
    max_length=96,
    return_tensors="np",
)
required = {item.name for item in session.get_inputs()}
feed = {name: encoded[name].astype(np.int64) for name in required}
logits = session.run(None, feed)[0]
label_id = int(logits.argmax(axis=-1)[0])
print(config.id2label[label_id])

Limitations

The intent vocabulary is project-specific and not a general-purpose intent taxonomy.
Confidence scores should be calibrated before using them as strict safety boundaries.
Inputs outside the five training intents are expected to fall back to chat.

Project documentation: PathFinderShip.