You can deploy this model and start earning money today!

mehmetPektas/
gpt-from-scratch-with-shakespeare

Name: gpt-from-scratch-with-shakespeare
Availability: InStock

Views

No views yet

GPT From Scratch (Character Level)

This model is a character-level Transformer language model trained on the Shakespeare dataset. It was built from scratch using PyTorch, following Andrej Karpathy's "Let's build GPT" tutorial.

Training Steps: 2000 iterations

Model Architecture

Embeddings: 384 dimensions
Heads: 4 heads
Layers: 4 transformer blocks
Block Size: 256 characters context window
Dropout: 0.2

Training results

Training Loss	Step	Validation Loss
4.1548	0	4.1548
1.8700	400	1.9857
1.5111	800	1.7069
1.3853	1200	1.6218
1.3089	1600	1.5623
1.2522	1999	1.5198

How to Use

To use this model, you need to download the model.py file to define the architecture and load the weights.

python

import torch
import pickle
from model import LanguageModel # Imports the class from the uploaded model.py

# 1. Load Tokenizer Mappings
with open('tokenizer_meta.pkl', 'rb') as f:
    meta = pickle.load(f)
stoi, itos = meta['stoi'], meta['itos']
encode = lambda s: [stoi[c] for c in s]
decode = lambda l: ''.join([itos[i] for i in l])

# 2. Initialize Model
device = 'cuda' if torch.cuda.is_available() else 'cpu'

# Note: Using the exact architecture from training
model = LanguageModel(
    vocab_size=len(meta['vocab_size']), # derived from meta
    n_embd=384,
    block_size=256,
    n_head=4, 
    n_layer=4
)

# 3. Load Weights
model.load_state_dict(torch.load('pytorch_model.pth', map_location=device))
model.to(device)
model.eval()

# 4. Generate Text
context = torch.zeros((1, 1), dtype=torch.long, device=device)
generated_ids = model.generate(context, max_new_tokens=500)[0].tolist()
print(decode(generated_ids))

mehmetPektas/gpt-from-scratch-with-shakespeare

GPT From Scratch (Character Level)

Model Architecture

Training results

How to Use

mehmetPektas/
gpt-from-scratch-with-shakespeare