Views
No views yet
| Property | Value |
|---|---|
| Architecture | MicroGPT (decoder-only transformer) |
| Parameters | 419,712 |
| Vocab Size | 74 (character-level) |
| Block Size | 128 tokens |
| Layers | 2 |
| Attention Heads | 4 |
| Embedding Dim | 128 |
| Training Steps | 500 |
| Best Val Loss | 2.2494189739227295 |
| Dataset | tinystories |
1# Clone the repo
2git clone https://huggingface.co/{{cookiecutter.repo_id if cookiecutter else 'your-username/micro-gpt-' + config['dataset']}}
3cd micro-gpt-tinystories
4
5# Install dependencies
6pip install -r requirements.txt
7
8# Generate text
9python inference.py --prompt "Once upon a time"1import torch
2from models.micro_gpt import MicroGPT
3
4# Load config
5import json
6with open('config.json') as f:
7 config = json.load(f)
8
9# Build model
10model = MicroGPT(
11 vocab_size=config['vocab_size'],
12 block_size=config['block_size'],
13 n_layer=config['n_layer'],
14 n_head=config['n_head'],
15 n_embd=config['n_embd'],
16 dropout=config['dropout'],
17)
18model.load_state_dict(torch.load('pytorch_model.bin', map_location='cpu'))
19model.eval()
20
21# Load tokenizer
22with open('tokenizer.json') as f:
23 tokenizer = json.load(f)
24
25# Encode prompt
26prompt = "Once upon a time"
27indices = [tokenizer['stoi'].get(c, 0) for c in prompt]
28input_ids = torch.tensor([indices], dtype=torch.long)
29
30# Generate
31output_ids = model.generate(input_ids, max_new_tokens=200, temperature=0.9, top_k=40)
32text = ''.join(tokenizer['itos'][str(i)] for i in output_ids[0].tolist())
33print(text)python run_model.py --train_model --arch micro_gpt --dataset tinystories --max_steps 500| File | Description |
|---|---|
models/ | Full model source code (MicroGPT architecture) |
pytorch_model.bin | Trained model weights |
config.json | Model hyperparameters |
tokenizer.json | Character-level tokenizer |
inference.py | Ready-to-use inference script |
requirements.txt | Python dependencies |