90,456 MLP fanfic stories scraped from fimfiction.net. No metadata is included, at least as columns, though story texts contain title and author.
Total tokens in directory 'stories': 1,946,219,954
Average tokens per file: 21,508.7578493673
Minimum tokens in a file: 11
Maximum tokens in a file: 5,083,037
Median tokens in a file: 5,969.0
Standard deviation of tokens in files: 74,087.33060019457
90th percentile of tokens in files: 44,269.5
Total tokens excluding top 10% longest files: 705,951,843… See the full description on the dataset page:
https://huggingface.co/datasets/trentmkelly/fimfiction-scrape.
