irds/

nyt

The nyt dataset, provided by the ir-datasets package. For more information about the dataset, see the documentation.

This dataset provides:

docs (documents, i.e., the corpus); count=1,864,661

This dataset is used by: nyt_trec-core-2017, nyt_wksup, nyt_wksup_train, nyt_wksup_valid

from datasets import load_dataset

docs = load_dataset('irds/nyt', 'docs') for record in docs: record # {'doc_id': ..., 'headline': ...… See the full description on the dataset page: https://huggingface.co/datasets/irds/nyt.