BetterWeb is a quality-filtered version of FineWeb that applies ensemble quality scoring inspired by state-of-the-art web data curation research.
BetterWeb combines three complementary quality signals, inspired by the findings of multiple research papers:
Trained on OpenHermes-2.5 + ELI5 (positive) vs RefinedWeb (negative)… See the full description on the dataset page:
https://huggingface.co/datasets/hynky/betterweb.
