Usage:
from datasets import load_dataset
corpus = load_dataset('Exr0n/wiki-entity-similarity', '2018thresh20corpus', split='train')
assert corpus[0] == {'article': 'A1000 road', 'link_text': 'A1000', 'is_same': 1}
pairs = load_dataset('Exr0n/wiki-entity-similarity', '2018thresh20pairs', split='train')
assert corpus[0] == {'article': 'Rhinobatos', 'link_text': 'Ehinobatos beurleni', 'is_same': 1}
assert len(corpus) == 4_793_180
Corpus (name=*corpus)… See the full description on the dataset page: https://huggingface.co/datasets/Exr0n/wiki-entity-similarity.