Views
No views yet
Note: This repo also containsnanogpt_slm_best.pth(nanoGPT architecture). The Raschka weights (gpt_slm_best.pth) use a different architecture with separate W_query/W_key/W_value projections and no weight tying.
1pip install torch tiktoken huggingface_hub
2python gpt_slm_pretrained_inference.pyask() in your own code1# Import loads the model automatically (one-time download from HuggingFace)
2from gpt_slm_pretrained_inference import ask, generate_text
3
4# Text completion
5print(ask("Once upon a time there was"))
6print()
7
8# Control generation
9print(ask(
10 "The meaning of life is",
11 temperature=1.0, # higher = more creative
12 top_k=100, # wider sampling pool
13 max_tokens=150 # longer output
14))
15print()
16
17# generate_text is an alias for ask
18print(generate_text("She opened the door and saw", max_tokens=200))
19print()1from huggingface_hub import hf_hub_download
2import torch
3
4model_path = hf_hub_download(
5 repo_id="nishantup/RaschkastyleGPT-pretrained-slm-163m",
6 filename="gpt_slm_best.pth"
7)
8
9from gpt_slm_pretrained_inference import GPTModel, BASE_CONFIG
10
11model = GPTModel(BASE_CONFIG)
12model.load_state_dict(torch.load(model_path, map_location="cpu"))
13model.eval()| Feature | gpt_slm_best.pth (Raschka) | nanogpt_slm_best.pth (nanoGPT) |
|---|---|---|
| Attention | Separate W_query, W_key, W_value | Combined c_attn |
| LayerNorm | scale/shift params | weight/bias params |
| MLP | FeedForward (Sequential) | MLP (c_fc/c_proj) |
| Config | Dict (BASE_CONFIG) | Dataclass (GPTConfig) |
| Weight tying | No | Yes (wte = lm_head) |
| forward() returns | logits | (logits, loss) tuple |
| KV Cache | Not included | Included (GPTKV) |
| Attribute | Value |
|---|---|
| Parameters | 163.2M |
| Architecture | Raschka GPTModel (12 layers, 12 heads, 768 dim) |
| Context length | 256 tokens |
| Tokenizer | tiktoken GPT-2 BPE (50,257 tokens) |
| Training data | 133 English fiction novels (37.5M tokens) |
| Framework | PyTorch |
| File | Description |
|---|---|
gpt_slm_best.pth | Pretrained weights (Raschka GPTModel) |
gpt_slm_pretrained_inference.py | Standalone inference script -- import and call ask() |
config_gpt_slm.json | Raschka model configuration |
ask() / generate_text() API Reference1ask(prompt, max_tokens=200, temperature=0.8, top_k=40)
2generate_text(prompt, max_tokens=200, temperature=0.8, top_k=40) # alias| Parameter | Default | Description |
|---|---|---|
prompt | (required) | Text to continue from |
max_tokens | 200 | Maximum tokens to generate |
temperature | 0.8 | 0.01 = near-greedy, 0.8 = balanced, 1.5 = creative |
top_k | 40 | Top-k filtering (None = no filtering) |
| Variant | Architecture | Repo / File |
|---|---|---|
| Pretrained (nanoGPT) | nanoGPT GPT class | nanogpt_slm_best.pth in this repo |
| Instruction-tuned (SFT) | nanoGPT GPT class | nishantup/nanogpt-slm-instruct |