Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
finepdfs_50BT-dclm_30BT-fineweb_edu_20BT – Dataset by HuggingFaceFW | AlphaNeural AI
You can deploy this model and start earning money today!
HuggingFaceFW
/
finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
like
0
en
odc-by
10M<n<100M
parquet
text
datasets
dask
polars
mlcroissant
us
pretraining
smol-data
Views
No views yet
Model card
Files and Versions
Community
API
FinePDFs 50BT + DCLM 30BT + FineWeb-Edu 20BT
A ~100 billion token pretraining mixture combining three high-quality English data sources in a 50-30-20 ratio. Part of the Smol-Data collection — tried and tested mixes for strong pretraining. Inspired by optimal dataset mixing.
Dataset Description
Component Source Tokens
FinePDFs 100BT FinePDFs ~50B
DCLM 100BT DCLM-Baseline 1.0 ~30B
FineWeb-Edu 100BT FineWeb-Edu ~20B
The schema is reduced to the… See the full description on the dataset page:
https://huggingface.co/datasets/HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
.
