A cleaned subset of the TinyStories dataset (Eldan & Li, 2023), keeping only GPT-4-generated stories. Adapted from this thread that pointed out many issues with the original data and proposed a cleaning process.
This cleaned dataset contains:
Stat
Value
Stories
2,732,634
Total characters
~2.19B
Min doc length
115 chars
Max doc length
4,433 chars
Median doc length
721 chars
Unique characters
74 (ASCII only)
