Is this your dataset? Claim it with the Hugging Face account that owns it.

Salesforce/
fineweb_deduplicated

Name: fineweb_deduplicated
Creator: Salesforce
License: https://spdx.org/licenses/ODC-By-1.0.html

Views

No views yet

TL;DR

Fineweb is a popular and high quality open dataset. This dataset is a deduplicated version of Fineweb - removing rows with duplicate text, collecting counts.

Motivation

Fineweb is an open text dataset intended for training language models. It's one of the highest quality and most popular open datasets available. It has been produced by a reputable AI lab - HuggingFace and has been downloaded tens of thousands of times. Fineweb dataset is 93.4 TB and has 15T… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/fineweb_deduplicated.