FineWeb-Edu base pretraining dataset prepared from karpathy/fineweb-edu-100b-shuffle.
text (string): original document text
tokens (int32): GPT-2 token count for text (add_special_tokens=False)
source dataset: karpathy/fineweb-edu-100b-shuffle
source split: train
tokenizer: gpt2
target train tokens: 700000000
actual train tokens: 700008540
uploaded rows: 680555
uploaded shards: 14… See the full description on the dataset page:
https://huggingface.co/datasets/Shekswess/fineweb-edu-700m.