1.1B+ high quality tokens from the internet, based on HuggingFaceTB/smollm-corpus's fineweb-edu-dedup subset as well as FineMath-4+.
Filtering process:
Step
Description
Rows
Stream dataset until 600K samples have been selected from SmolLM Corpus
Keep only items with score >= 3.5
600,000
Remove items with length > 50,000
Filter items exceeding 50,000 characters in length
597,142
