This is a filtered version of DCLM dataset using FineWeb-Edu educational quality classifier. We annotate each web page based on the educational quality
on a scale from 0 to 5 and only keep samples with a score higher than 2. This dataset is intended for small language models training and was used to train SmolLM2-135M and SmolLM2-360M.
Note: As show in the performance section, we find that further filtering the dataset to only keep samples with… See the full description on the dataset page:
https://huggingface.co/datasets/HuggingFaceTB/dclm-edu.
