Is this your dataset? Claim it with the Hugging Face account that owns it.

Avelina/
smollm-corpus-cleaned

Name: smollm-corpus-cleaned
Creator: Avelina
License: https://spdx.org/licenses/ODC-By-1.0.html

Views

No views yet

SmolLM-Corpus: Now shuffled and sharded (and Cleaned)!

This is a version of the SmolLM-Corpus where the 3 subsets have been interleved, shuffled and sharded as 23698 jsonl.zst files for easy streaming! The dataset is comprised of the cosmopedia-v2 and fineweb-edu-dedup subsets from the original SmolLM-Corpus repo, with the python-edu subset being pulled from my python-edu-cleaned repo.

Dataset Structure

The dataset is split into 24 subdirectories, with the first 23… See the full description on the dataset page: https://huggingface.co/datasets/Avelina/smollm-corpus-cleaned.