Views
No views yet
microsoft/deberta-v3-base. It was trained on Ms-Marco using loss hingeLoss as part of a reproducibility paper for training cross encoders: "Reproducing and Comparing Distillation Techniques for Cross-Encoders", see the paper for more details.1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("xpmir/cross-encoder-DeBERTav3-Hinge")
5model = AutoModelForSequenceClassification.from_pretrained("xpmir/cross-encoder-DeBERTav3-Hinge")
6
7features = tokenizer("What is experimaestro ?", "Experimaestro is a powerful framework for ML experiments management...", padding=True, truncation=True, return_tensors="pt")
8
9model.eval()
10with torch.no_grad():
11 scores = model(**features).logits
12 print(scores)1000 documents retrieved by naver/splade-v3-distilbert.| dataset | RR@10 | nDCG@10 |
|---|---|---|
| msmarco_dev | 36.22 | 42.66 |
| trec2019 | 96.12 | 70.14 |
| trec2020 | 91.21 | 67.87 |
| fever | 75.48 | 75.39 |
| arguana | 14.36 | 21.38 |
| climate_fever | 24.22 | 18.07 |
| dbpedia | 67.99 | 38.75 |
| fiqa | 46.11 | 37.81 |
| hotpotqa | 75.45 | 57.94 |
| nfcorpus | 49.05 | 28.70 |
| nq | 50.32 | 55.25 |
| quora | 61.89 | 64.80 |
| scidocs | 26.07 | 14.97 |
| scifact | 66.01 | 68.71 |
| touche | 56.51 | 33.08 |
| trec_covid | 91.57 | 73.45 |
| robust04 | 64.27 | 42.59 |
| lotte_writing | 66.27 | 57.80 |
| lotte_recreation | 61.18 | 55.67 |
| lotte_science | 46.46 | 38.88 |
| lotte_technology | 54.78 | 45.97 |
| lotte_lifestyle | 73.88 | 64.68 |
| Mean In Domain | 74.52 | 60.22 |
| BEIR 13 | 54.23 | 45.25 |
| LoTTE (OOD) | 61.14 | 50.93 |