hotchpotch/

wikipedia-multilingual-synthetic-ir-query

This dataset contains multilingual Wikipedia-derived synthetic query-document pairs for information retrieval training. It was created with the query-crafter-multilingual model, which generates search-like queries from Wikipedia text. The current release contains two different retrieval settings:

short_doc: pairs of (query, short document) long_doc: pairs of (query, long document)

These two subsets are not generated in the same way… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/wikipedia-multilingual-synthetic-ir-query.