Is this your dataset? Claim it with the Hugging Face account that owns it.

HuggingFaceFW/
finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT

Name: finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Creator: HuggingFaceFW
License: https://spdx.org/licenses/ODC-By-1.0.html

Views

No views yet

FinePDFs-Edu 50BT + DCLM 30BT + FineWeb-Edu 20BT

A ~100 billion token pretraining mixture combining three high-quality English data sources in a 50-30-20 ratio, using the educational subset of FinePDFs. Part of the Smol-Data collection — tried and tested mixes for strong pretraining. Inspired by optimal dataset mixing.

	Dataset Description

Component Source Tokens

FinePDFs-Edu 100BT FinePDFs-Edu ~50B

DCLM 100BT DCLM-Baseline 1.0 ~30B

FineWeb-Edu 100BT… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT.