What it is: Small GPT (~50M params) trained on TinyStories for text generation
Credits: Architecture from nanoGPT (Karpathy), training from [tutorial], data: TinyStories (Ronen Eldan et al.)
How to use:
Download pytorch_model.pt
Build the same GPT(config) (vocab 50257, block_size 128, 6 layers, 6 heads, 384 dim)
Load: model.load_state_dict(torch.load("pytorch_model.pt", map_location="cpu"))
Tokenizer: tiktoken gpt2
language: en
license: mit
tags:
- text-generation
- tiny-stories
