You can deploy this model and start earning money today!

dmrau/
bow-bert

Name: bow-bert
Rating: 2.5 (10 reviews)

Views

No views yet

This model was trained with the Rankers Library please check https://github.com/davidmrau/rankers#Evaluation to learn how to use the model.

This Cross Encoder Ranking Model is invariant to the input order by removing the position embeddings (by being set to zero) during fine-tuning. It was trained on the official MS MARCO training triples using the following training parameters:

batch size: 128
learning rate 3e-6
Optimizer: Adam

Example on how to load and use BOW-BERT:

python
1from transformers import AutoModelForSequenceClassification, AutoTokenizer 2 3# load model 4model = AutoModelForSequenceClassification.from_pretrained('dmrau/bow-bert') 5# load tokenizer 6tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased') 7 8# tokenize query and passage and concatenate them 9inp = tokenizer(['this is a query','query a is this'], ['this is a passage', 'passage a is this'], return_tensors='pt') 10# get estimated score 11print('score', model(**inp).logits[:, 1]) 12 13### outputs identical scores for different 14### word orders as the model is order invariant: 15# scores: [-2.9463, -2.9463]

Cite us:

@article{rau2022role, title={The Role of Complex NLP in Transformers for Text Ranking?}, author={Rau, David and Kamps, Jaap}, journal={arXiv preprint arXiv:2207.02522}, year={2022} }