Views
No views yet
MoritzLaurer/mDeBERTa-v3-base-mnli-xnli (MIT).scripts/export_xnli_onnx.py in hikma-mirsad (optimum main_export, opset 17; INT8 dynamic quant for CPU, FP16 for GPU)."This text is about <label>.". The gateway takes softmax over the entailment vs
contradiction logits (neutral is ignored); the entailment probability is the on-topic score.internal/topicalml/scorer.go) depends on this contract, validated at export:input_ids + attention_mask (mDeBERTa-v3 has type_vocab_size=0, so no token_type_ids).0=entailment, 1=neutral, 2=contradiction.P(entailment | "about weapons") = 0.998, P(entailment | "about cooking") = 0.001.| Path | Precision | Use |
|---|---|---|
onnx/int8/model_quantized.onnx | INT8 dynamic-quant | CPU (edge/hub hot path) |
onnx/fp16/model.onnx | FP16 | GPU (CUDA execution provider) |
onnx/fp32/model.onnx | FP32 | source for re-quantization |
onnx/int8/tokenizer.json | — | fast tokenizer (loaded by the gateway) |