You can deploy this model and start earning money today!

ethanker/
math-reasoning-packed-1024

Views

No views yet

math-reasoning-packed-1024

Packed token dataset ready for GPT-style pretraining.

Dataset Description

This dataset contains pre-tokenized and packed sequences optimized for efficient transformer training.

Dataset Statistics

Encoding: cl100k_base Vocabulary Size: 100,277 Sequence Length: 1024

Training Split

Documents: 684,238 Total Tokens: 239,019,783 Sequences: 233,417 Shards: 3

Validation Split

Documents: 13,961 Total Tokens: 4,875… See the full description on the dataset page: https://huggingface.co/datasets/ethanker/math-reasoning-packed-1024.