allenai/

ms2_dense_max

This is a copy of the MS^2 dataset, except the input source documents of its validation split have been replaced by a dense retriever. The retrieval pipeline used:

query: The background field of each example corpus: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract. retriever: facebook/contriever-msmarco via PyTerrier with default settings top-k strategy: "max", i.e. the number of documents retrieved, k, is set as… See the full description on the dataset page: https://huggingface.co/datasets/allenai/ms2_dense_max.