Views
No views yet
google/electra-base-discriminator. It was trained on Ms-Marco using loss infoNCE as part of a reproducibility paper for training cross encoders: "Reproducing and Comparing Distillation Techniques for Cross-Encoders", see the paper for more details.1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("xpmir/cross-encoder-ELECTRA-infoNCE")
5model = AutoModelForSequenceClassification.from_pretrained("xpmir/cross-encoder-ELECTRA-infoNCE")
6
7features = tokenizer("What is experimaestro ?", "Experimaestro is a powerful framework for ML experiments management...", padding=True, truncation=True, return_tensors="pt")
8
9model.eval()
10with torch.no_grad():
11 scores = model(**features).logits
12 print(scores)1000 documents retrieved by naver/splade-v3-distilbert.| dataset | RR@10 | nDCG@10 |
|---|---|---|
| msmarco_dev | 41.06 | 47.71 |
| trec2019 | 95.35 | 75.02 |
| trec2020 | 95.52 | 75.06 |
| fever | 80.23 | 80.36 |
| arguana | 20.60 | 30.72 |
| climate_fever | 29.69 | 22.16 |
| dbpedia | 75.94 | 45.49 |
| fiqa | 50.16 | 41.51 |
| hotpotqa | 88.46 | 71.00 |
| nfcorpus | 58.28 | 35.65 |
| nq | 55.53 | 60.49 |
| quora | 78.10 | 80.39 |
| scidocs | 29.06 | 16.48 |
| scifact | 67.34 | 70.36 |
| touche | 64.08 | 35.61 |
| trec_covid | 93.17 | 70.92 |
| robust04 | 72.21 | 49.80 |
| lotte_writing | 72.46 | 63.84 |
| lotte_recreation | 63.54 | 57.97 |
| lotte_science | 48.91 | 40.97 |
| lotte_technology | 57.57 | 47.87 |
| lotte_lifestyle | 74.62 | 65.47 |
| Mean In Domain | 77.31 | 65.93 |
| BEIR 13 | 60.82 | 50.86 |
| LoTTE (OOD) | 64.89 | 54.32 |