Aggregate statistics computed using Polars streaming on the HuggingFaceFW/finepdfs-edu dataset.
Processed 23,023,372 documents in 2608.95 seconds.
Speed comes from Polars only reading metadata columns (not the text column),
thanks to Parquet's columnar format… See the full description on the dataset page:
https://huggingface.co/datasets/davanstrien/finepdfs-eng-stats.