PAWN-Large
PAWN (Playstyle-Agnostic World-model Network for Chess) is a causal transformer trained on random chess games. It learns legal moves, board state representations, and game dynamics purely from uniformly random legal move sequences -- no strategic play, no hand-crafted features, no external game databases.
This is the large variant (~66.9M parameters). PAWN is designed as a frozen backbone for parameter-efficient finetuning into player models with arbitrary playstyles.
GitHub Repository -- full source code, training scripts, adapter implementations, and documentation.
All Variants
A previous generation of PAWN backbones (
pawn-{small,base,large}-legacy) used a 4,278-token coordinate vocabulary, a 256-token context window, and outcome conditioning. They are still available on HuggingFace; see
docs/LEGACY.md for the full story.
Headline Metrics
These come from the published model.safetensors (step 195,000 out of 200,000 — the best 5,000-step-cadence checkpoint by val loss), measured on a fresh validation set of random games.
| Metric | Value |
|---|
| Game completion rate | 99.76% |
| Per-move legal rate | 99.9990% |
| Late-game legal rate | 99.9996% |
| Top-1 accuracy | 8.63% |
| Top-5 accuracy | 35.56% |
| Val loss | 2.865 |
| Val perplexity | 17.55 |
Game completion rate is the share of validation games in which
every prediction along one side's plies was a legal move. The measurement is
non-autoregressive: at each ply the model is shown the true ground-truth history and asked for that side's next move, and an illegal prediction at any ply forfeits the game. Errors do not corrupt subsequent positions — each prediction is independent given the true history. Autoregressive game completion has not been measured for these checkpoints and could be higher or lower; see the
game completion section of the architecture doc for the full definition. Game completion rate is a much stricter metric than per-move legal rate, and is the main signal that separates capacity between sizes.
| Compound-legality detail | Value |
|---|
| Average plies completed per game | 349 |
| Average % of game completed | 99.88% |
| Median forfeit ply (when forfeit) | 153 |
Accuracy ceiling
PAWN is trained on uniformly random chess games. At each position with N legal moves, the next move is drawn uniformly, so the Bayes-optimal predictor that does not know the game's outcome can do no better than 1/N at that position. Averaged over the position distribution induced by random games of up to 512 plies, the top-1 ceiling is
E[1/N_legal] ≈ 8.43% (95% CI [8.41%, 8.45%], computed over 50,000 fresh random games — see
docs/ACCURACY_CEILING.md).
This model's top-1 accuracy of 8.63% is 102% of that ceiling — i.e., essentially at the limit of what any predictor can achieve on this task without outcome information.
Probe Results
Linear probes trained on frozen hidden states measure how well the model's internal representations encode board-level features. The model is never explicitly told about pieces, sides, or rules — these representations emerge purely from next-token prediction on random games.
| Probe | Metric | Description |
|---|
| Piece type | 92.2% | Per-square piece type (13 classes x 64 squares) |
| Side to move | 100.0% | Whose turn it is |
| Is check | 94.9% | Whether the side to move is in check |
| Castling rights | 99.3% | KQkq castling availability |
| En passant square | 99.9% | En passant target square (64 + none) |
| Material count | R² 0.83 (MAE 4.2) | Piece counts per type per color |
| Legal move count | R² 0.70 (MAE 5.0) | Number of legal moves available |
| Halfmove clock | R² 0.46 (MAE 10.8) | Plies since last capture or pawn move |
| Game phase | 95.9% | Opening / middlegame / endgame |
Diagnostic Results
Edge-case diagnostics measure the model's legal move rate in specific tactical situations.
| Category | Positions | Legal Rate |
|---|
| In check | 10000 | 99.6% |
| Double check | 10000 | 98.1% |
| Pin restricts movement | 10000 | 99.5% |
| En passant available | 10000 | 99.8% |
| Castling legal (kingside) | 10000 | 99.9% |
| Castling legal (queenside) | 10000 | 99.8% |
| Castling blocked by check | 10000 | 99.8% |
| Promotion available | 10000 | 99.8% |
| Checkmate (terminal) | 10000 | 96.4% |
| Stalemate (terminal) | 10000 | 99.1% |
Architecture
| Parameter | Value |
|---|
| Architecture | Decoder-only transformer |
| d_model | 640 |
| Layers | 10 |
| Attention heads | 8 |
| Head dimension | 80 |
| d_ff | 2560 |
| Parameters | ~66.9M |
| Vocabulary | 1,980 tokens (1,968 searchless_chess actions + 1 PAD + 11 outcome tokens) |
| Context length | 512 tokens |
| Normalization | Pre-norm RMSNorm |
| FFN | SwiGLU (4x expansion) |
| Positional encoding | Rotary (RoPE, base 10000) |
| Embeddings | Factored (src + dst + promo) |
| Dropout | 0.0 |
Training Details
| Parameter | Value |
|---|
| Training data | On-the-fly uniformly random legal games (no external dataset) |
| Objective | Next-token cross-entropy (non-padding positions only) |
| Outcome conditioning | Disabled (prepend_outcome=False) — pure moves, no outcome leakage |
| Total steps | 200,000 |
| Batch size | 256 |
| Total training sequences | 51,200,000 (= total steps × batch size; the published checkpoint is the best 5K-cadence step by val loss, at step 195,000 ≈ 49,920,000 sequences) |
| Max ply per example | 512 |
| Learning rate | 0.0003 (cosine decay with 10,000-step warmup) |
| Optimizer | AdamW (weight decay 0.01) |
| Precision | Mixed (AMP) |
Usage
Loading the model
1import torch
2from safetensors.torch import load_file
3from pawn.config import CLMConfig
4from pawn.model import PAWNCLM
5
6cfg = CLMConfig.large()
7model = PAWNCLM(cfg).cuda().eval()
8weights = load_file("model.safetensors", device="cuda")
9model.load_state_dict(weights)
Or load directly from HuggingFace:
1from pawn.checkpoint import load_backbone_weights
2from pawn.config import CLMConfig
3from pawn.model import PAWNCLM
4
5weights, config = load_backbone_weights("thomas-schweich/pawn-large")
6cfg = CLMConfig.large()
7model = PAWNCLM(cfg).eval()
8model.load_state_dict(weights)
Finetuning with an adapter
1uv run python scripts/train.py --run-type adapter --strategy bottleneck \
2 --checkpoint thomas-schweich/pawn-large \
3 --pgn thomas-schweich/pawn-lichess-full \
4 --bottleneck-dim 32 --lr 1e-4 --local-checkpoints
Acknowledgments
PAWN builds on ideas and tools from the following projects and publications:
| Component | Reference |
|---|
| Transformer | Vaswani et al., "Attention Is All You Need", NeurIPS 2017 |
| RMSNorm | Zhang & Sennrich, "Root Mean Square Layer Normalization", NeurIPS 2019 |
| RoPE | Su et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding", 2021 |
| SwiGLU | Shazeer, "GLU Variants Improve Transformer", 2020 |
| AdamW | Loshchilov & Hutter, "Decoupled Weight Decay Regularization", ICLR 2019 |
| Cosine schedule | Loshchilov & Hutter, "SGDR: Stochastic Gradient Descent with Warm Restarts", ICLR 2017 |
| Mixed precision | Micikevicius et al., "Mixed Precision Training", ICLR 2018 |
| Bottleneck adapters | Houlsby et al., "Parameter-Efficient Transfer Learning for NLP", ICML 2019 |
| LoRA | Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models", ICLR 2022 |
| FiLM | Perez et al., "FiLM: Visual Reasoning with a General Conditioning Layer", AAAI 2018 |
| RoSA | Nikdan et al., "RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation", 2024 |
| Linear probes | Alain & Bengio, "Understanding Intermediate Layers Using Linear Classifier Probes", ICLR Workshop 2017 |
| Searchless Chess (action vocab) | Ruoss et al., "Amortized Planning with Large-Scale Transformers: A Case Study on Chess", 2024 |
| MAIA | McIlroy-Young et al., "Aligning Superhuman AI with Human Behavior: Chess as a Model System", KDD 2020 |
| AlphaZero | Silver et al., "A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play", Science 2018 |
| Leela Chess Zero | github.com/LeelaChessZero/lc0 |
| shakmaty | github.com/niklasf/shakmaty |
| PyO3 | github.com/PyO3/pyo3 |
| Lichess | lichess.org / database.lichess.org |
Citation
1@software{schweich2026pawn,
2 author = {Schweich, Thomas},
3 title = {{PAWN}: Playstyle-Agnostic World-model Network for Chess},
4 year = {2026},
5 url = {https://github.com/thomas-schweich/PAWN},
6 license = {Apache-2.0}
7}
License
