Views
No views yet
microsoft/deberta-v3-base. It was trained on Ms-Marco using loss ADR as part of a reproducibility paper for training cross encoders: "Reproducing and Comparing Distillation Techniques for Cross-Encoders", see the paper for more details.1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("xpmir/cross-encoder-DeBERTav3-ADR-MSE")
5model = AutoModelForSequenceClassification.from_pretrained("xpmir/cross-encoder-DeBERTav3-ADR-MSE")
6
7features = tokenizer("What is experimaestro ?", "Experimaestro is a powerful framework for ML experiments management...", padding=True, truncation=True, return_tensors="pt")
8
9model.eval()
10with torch.no_grad():
11 scores = model(**features).logits
12 print(scores)1000 documents retrieved by naver/splade-v3-distilbert.| dataset | RR@10 | nDCG@10 |
|---|---|---|
| msmarco_dev | 35.11 | 41.61 |
| trec2019 | 93.60 | 74.47 |
| trec2020 | 95.06 | 71.98 |
| fever | 82.73 | 82.03 |
| arguana | 14.10 | 21.77 |
| climate_fever | 27.96 | 20.23 |
| dbpedia | 73.43 | 42.60 |
| fiqa | 44.04 | 36.59 |
| hotpotqa | 75.64 | 58.04 |
| nfcorpus | 51.21 | 30.66 |
| nq | 53.19 | 58.19 |
| quora | 71.59 | 74.08 |
| scidocs | 26.33 | 14.86 |
| scifact | 67.32 | 70.28 |
| touche | 64.97 | 35.00 |
| trec_covid | 94.33 | 78.61 |
| robust04 | 70.72 | 46.79 |
| lotte_writing | 67.14 | 58.43 |
| lotte_recreation | 61.62 | 56.57 |
| lotte_science | 48.52 | 40.41 |
| lotte_technology | 55.36 | 46.19 |
| lotte_lifestyle | 74.52 | 65.28 |
| Mean In Domain | 74.59 | 62.69 |
| BEIR 13 | 57.45 | 47.92 |
| LoTTE (OOD) | 62.98 | 52.28 |