Views
No views yet
standalone_transformer_lm.GPT) and SentencePiece tokenization — no Hugging Face Transformers dependency.standalone_transformer_lm.pytokenizer.model)| Step | Val Loss |
|---|---|
| 1,000 | 5.6011 |
| 2,000 | 4.8598 |
| 5,000 | 4.2239 |
| 10,000 | 3.9756 |
| 15,000 | 3.8608 |
| 20,000 | 3.7984 |
standalone_transformer_lm.py and SentencePiece to run.transformers.AutoModelForCausalLM.1import torch, sentencepiece as spm
2from standalone_transformer_lm import GPT, GPTConfig
3
4# Load checkpoint & config
5ckpt = torch.load("ckpt_best.pt", map_location="cpu")
6cfg = GPTConfig(**ckpt["config"])
7
8# Init model & load weights
9model = GPT(cfg).eval()
10model.load_state_dict(ckpt["model"])
11
12# Load tokenizer
13sp = spm.SentencePieceProcessor()
14sp.load("tokenizer.model")
15
16# Encode prompt
17ids = torch.tensor([sp.encode("Dubai is", out_type=int)])
18
19# Generate text
20out = model.generate(ids, max_new_tokens=40)
21print(sp.decode(out[0].tolist()))1import torch, sentencepiece as spm
2from standalone_transformer_lm import GPT, GPTConfig
3
4ckpt = torch.load("ckpt_best.pt", map_location="cpu")
5cfg = GPTConfig(**ckpt["config"])
6model = GPT(cfg).eval()
7model.load_state_dict(ckpt["model"])
8
9sp = spm.SentencePieceProcessor()
10sp.load("tokenizer.model")
11
12prompt = "when a man goes to fishing"
13ids = torch.tensor([sp.encode(prompt, out_type=int)])
14
15# Manual repetition control
16out = model.generate(
17 ids,
18 max_new_tokens=100,
19 temperature=0.7, # Lower temp = more focused
20 top_k=50, # Top-K sampling
21 top_p=0.9, # Nucleus sampling
22 repetition_penalty=1.2, # Penalize repeats
23 no_repeat_ngram_size=3, # Block repeating trigrams
24)
25print(sp.decode(out[0].tolist()))repetition_penalty to 1.2–1.5no_repeat_ngram_size=3 or highertop_k and top_p for better sampling varietytemperature for more deterministic completions