irds/

clueweb09_ko

The clueweb09/ko dataset, provided by the ir-datasets package. For more information about the dataset, see the documentation.

This dataset provides:

docs (documents, i.e., the corpus); count=18,075,141

from datasets import load_dataset

docs = load_dataset('irds/clueweb09_ko', 'docs') for record in docs: record # {'doc_id': ..., 'url': ..., 'date': ..., 'http_headers': ..., 'body': ..., 'body_content_type': ...}… See the full description on the dataset page: https://huggingface.co/datasets/irds/clueweb09_ko.