jbduran/

bartholomew-dataset-v2

The second version of the BART pretraining corpus, focused on stripping low-quality text — boilerplate and OCR corruption — out of v1.

Documents 149,745 (93.44% of v1)

Characters 106,274,384,672 (89.50% of v1)

Tokens ~24B (estimated)

Shards 473 (one per v1 shard, same basename)

Source BART Dataset v1

Schema single string column text

Lineage — three cumulative filtering stages over the same corpus: Institutional Books 1.0 → v1 → v2 → v3… See the full description on the dataset page: https://huggingface.co/datasets/jbduran/bartholomew-dataset-v2.