You can deploy this model and start earning money today!

achavan1211/
shakespeare-gpt

Name: shakespeare-gpt
Availability: InStock

Views

No views yet

Shakespeare GPT

A small character-level GPT language model (~10.8M parameters) trained from scratch on the Tiny Shakespeare dataset following Andrej Karpathy's "Let's build GPT" tutorial.

Model Details

Architecture: decoder-only Transformer
Vocab size: 65 (character-level)
Embedding dim: 384
Number of heads: 6
Number of layers: 6
Context length (block size): 256
Total parameters: 10.79M

Training

Dataset: Tiny Shakespeare
Optimizer: AdamW (lr=0.001)
Iterations: 5000
Final train loss: 0.8415
Final val loss: 1.5810

Usage

This is a custom PyTorch model. See model.py for the architecture, config.json for hyperparameters, and vocab.json for the character-level tokenizer mappings.

python

import json, torch
from huggingface_hub import hf_hub_download
from model import GPTLanguageModel


config = json.load(open(hf_hub_download('achavan1211/shakespeare-gpt', 'config.json')))
vocab = json.load(open(hf_hub_download('achavan1211/shakespeare-gpt', 'vocab.json')))
stoi, itos = vocab['stoi'], {int(k): v for k, v in vocab['itos'].items()}
encode = lambda s: [stoi[c] for c in s]
decode = lambda l: ''.join(itos[i] for i in l)

model = GPTLanguageModel(**config)
model.load_state_dict(torch.load(hf_hub_download('achavan1211/shakespeare-gpt', 'pytorch_model.bin'), map_location='cpu'))
model.eval()

# Generate text
context = torch.zeros((1, 1), dtype=torch.long)
print(decode(model.generate(context, max_new_tokens=500)[0].tolist()))