Given a question and paragraph, can the question be answered by the paragraph? The models have been trained on the GLUE QNLI dataset, which transformed the SQuAD dataset into an NLI task.

Performance

For performance results of this model, see [SBERT.net Pre-trained Cross-Encoder][https://www.sbert.net/docs/pretrained_cross-encoders.html].

Usage

Pre-trained models can be used like this:

python

from sentence_transformers import CrossEncoder

model = CrossEncoder('cross-encoder/qnli-distilroberta-base')
scores = model.predict([('Query1', 'Paragraph1'), ('Query2', 'Paragraph2')])

#e.g.
scores = model.predict([('How many people live in Berlin?', 'Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.'), ('What is the size of New York?', 'New York City is famous for the Metropolitan Museum of Art.')])

Usage with Transformers AutoModel

You can use the model also directly with Transformers library (without SentenceTransformers library):

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained('cross-encoder/qnli-distilroberta-base')
tokenizer = AutoTokenizer.from_pretrained('cross-encoder/qnli-distilroberta-base')

features = tokenizer(['How many people live in Berlin?', 'What is the size of New York?'], ['Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'],  padding=True, truncation=True, return_tensors="pt")

model.eval()
with torch.no_grad():
    scores = torch.nn.functional.sigmoid(model(**features).logits)
    print(scores)