This is a processed dump of the English Wikipedia from 2021-01-01. Each page has been splitted into paragraphs as they appear in the text. Lists, tables and headlines had been removed. In total it has 38,080,804 passages.
Further, each article contain meta-data on the number of languages this article exists in and on the number of views this article received over a 1 year period.
The articles are sorted from most popular (most languages available, most… See the full description on the dataset page:
https://huggingface.co/datasets/vocab-transformers/wiki-en-passages-20210101.
