irds/

mmarco_v2_id

The mmarco/v2/id dataset, provided by the ir-datasets package. For more information about the dataset, see the documentation.

This dataset provides:

docs (documents, i.e., the corpus); count=8,841,823

This dataset is used by: mmarco_v2_id_dev, mmarco_v2_id_train

from datasets import load_dataset

docs = load_dataset('irds/mmarco_v2_id', 'docs') for record in docs: record # {'doc_id': ..., 'text': ...}

Note that… See the full description on the dataset page: https://huggingface.co/datasets/irds/mmarco_v2_id.