This checkpoint is a causal-LM yes/no scorer, saved as Qwen3ForCausalLM (Qwen3MoeForCausalLM for MoE). The relevance score of a (query, passage) pair is the yes/no log-odds at the final prompt token:

score = logit(" yes") − logit(" no")

evaluated right after the prompt suffix "\nIs the passage relevant to the query?". The prompt is "query: {{query}} passage: {{title}} {{text}}\nIs the passage relevant to the query?" and no EOS token is appended.

This differs from the HF sequence-classification rerankers in this release (*-hf, *-lora-hf), which score a single regression logit at EOS.

Usage

Score with Tevatron's reranker eval backend (see the Tevatron repo), or directly:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

name = "brutusxu/tevatron3-reranker-8b-distill"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.bfloat16).cuda().eval()
yes_id = tok.encode(" yes", add_special_tokens=False)[-1]
no_id  = tok.encode(" no",  add_special_tokens=False)[-1]
prompt = ("query: what is the capital of france "
          "passage: Paris is the capital of France.\n"
          "Is the passage relevant to the query?")
ids = tok(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    logits = model(**ids).logits[0, -1]
print((logits[yes_id] - logits[no_id]).item())

Citation

If you use this model, please cite the Tevatron toolkit.

utahnlp/tevatron3-reranker-8b-distill

tevatron3-reranker-8b-distill

Scoring contract (important)

Usage

Citation

utahnlp/
tevatron3-reranker-8b-distill