Pre-tokenized FineWeb-Edu
(sample/100BT), packed into fixed-length sequences and stored as
ArrayRecord shards for zero-overhead
streaming with Grain. No on-the-fly tokenization
at train time — you read int32 tokens straight off disk.
Tokenizer: google/gemma-4-12B-it (vocab size 262144). Documents are
separated by the EOS token id 1.
Packing: the token stream is… See the full description on the dataset page:
https://huggingface.co/datasets/mlnomad/fineweb-edu-gemma4-1024.
