An ONNX-converted, BYOM-ready copy of mixedbread-ai/mxbai-rerank-xsmall-v1: given a (query, passage) pair, it outputs a single relevance score for that pair.

Retrieval pipelines typically run in two stages. The first stage -- a vector search over an index such as HNSW, or classical keyword search -- scans the whole corpus and returns a shortlist (say, the top 20-100 candidates). It's fast because it embeds the query and each document independently and only ever compares the resulting vectors; the two texts never interact. A reranker is a cross-encoder: it reads the query and one candidate together, token by token, and produces a single relevance score for that pair. That joint attention makes it far more accurate than the first-stage vector search, but also too expensive to run over an entire corpus. The standard pattern is therefore to let cheap first-stage retrieval narrow things down, then use a reranker like this one to re-score and re-order just that shortlist.

relevance_score is the model's raw logit -- sorting by it directly already gives the correct ranking, since sigmoid is monotonic. If you need an actual [0, 1] probability rather than just a ranking, compute it in SQL instead of Python: 1 / (1 + EXP(-emb_0)).

A typical pipeline: embed your corpus once with an embedding model, then at query time use VectorDistance to get a shortlist of candidates by cosine distance, and rerank only that shortlist with this model.

ONNXEmbeddings is built to map one text column to a fixed-size float vector -- it has no native concept of a (query, passage) pair, and doesn't care that its output is semantically an "embedding": any graph with the right tensor names/shapes runs. This repo reuses it as a generic text-to-number function instead: query and passage are concatenated into a single string before tokenization, and the model's one classification logit comes back through the (1-value) sentence_embedding output.

This is only safe because DeBERTa-v3 is configured with type_vocab_size=0 -- its exported graph never uses token_type_ids to mark the query/passage boundary -- confirmed by testing the concatenated score against the model's true pair-encoded score (AutoModelForSequenceClassification, real two-argument tokenizer call), both locally and against a live Vantage instance:

If you plan to convert a different reranker with this same recipe, check first whether its exported graph actually declares/uses token_type_ids as an input -- if it does, this concat trick is not valid for that model.

mixedbread-ai/mxbai-rerank-xsmall-v1 already ships a ready-made ONNX export (onnx/model.onnx, fp32, opset 12) -- no re-export via optimum was needed. The only change required was renaming the graph's native output tensor (logits) to sentence_embedding, since ONNXEmbeddings's model_output_tensor argument only accepts the literal values sentence_embedding or token_embeddings. See convert.py for the conversion steps, and test_local.py for the ONNX-vs-PyTorch parity check above.

DISCLAIMER: The content herein ("Content") is provided "AS IS" and is not covered by any Teradata Operations, Inc. and its affiliates ("Teradata") agreements. Its listing here does not constitute certification or endorsement by Teradata.

To the extent any of the Content contains or is related to any artificial intelligence ("AI") or other language learning models ("Models") that interoperate with the products and services of Teradata, by accessing, bringing, deploying or using such Models, you acknowledge and agree that you are solely responsible for ensuring compliance with all applicable laws, regulations, and restrictions governing the use, deployment, and distribution of AI technologies. This includes, but is not limited to, AI Diffusion Rules, European Union AI Act, AI-related laws and regulations, privacy laws, export controls, and financial or sector-specific regulations.

While Teradata may provide support, guidance, or assistance in the deployment or implementation of Models to interoperate with Teradata's products and/or services, you remain fully responsible for ensuring that your Models, data, and applications comply with all relevant legal and regulatory obligations. Our assistance does not constitute legal or regulatory approval, and Teradata disclaims any liability arising from non-compliance with applicable laws.

You must determine the suitability of the Models for any purpose. Given the probabilistic nature of machine learning and modeling, the use of the Models may in some situations result in incorrect output that does not accurately reflect the action generated. You should evaluate the accuracy of any output as appropriate for your use case, including by using human review of the output.