irds/

beir_msmarco

The beir/msmarco dataset, provided by the ir-datasets package. For more information about the dataset, see the documentation.

This dataset provides:

docs (documents, i.e., the corpus); count=8,841,823 queries (i.e., topics); count=509,962

This dataset is used by: beir_msmarco_dev, beir_msmarco_test, beir_msmarco_train

from datasets import load_dataset

docs = load_dataset('irds/beir_msmarco', 'docs') for record in… See the full description on the dataset page: https://huggingface.co/datasets/irds/beir_msmarco.