allenai/

ms2_sparse_max

This is a copy of the MS^2 dataset, except the input source documents of its validation split have been replaced by a sparse retriever. The retrieval pipeline used:

query: The background field of each example corpus: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract. retriever: BM25 via PyTerrier with default settings top-k strategy: "max", i.e. the number of documents retrieved, k, is set as the maximum number of… See the full description on the dataset page: https://huggingface.co/datasets/allenai/ms2_sparse_max.