You can deploy this model and start earning money today!

justjuu/
tinystories-gpt2-shards

Views

No views yet

storyGPT GPT-2 Token Shards

This repository contains the GPT-2-tokenized binary shards used to train storyGPT. They were prepared from the TinyStories dataset.

	Files

Split File Size Tokens

train train.bin 0.88 GB 473,992,236

validation validation.bin 0.01 GB 4,765,918

Each file is a contiguous one-dimensional array of little-endian NumPy uint16 token IDs. Stories are separated by GPT-2 end-of-text token 50256.

	Download and Load… See the full description on the dataset page: https://huggingface.co/datasets/justjuu/tinystories-gpt2-shards.