Chessformer v0
A ~50M parameter transformer trained to imitate human chess play across the full Elo spectrum, conditioned on player Elo, remaining clock time, and time increment.
44.4% top-1 joint move accuracy on a held-out, Elo-balanced validation set (predicting the exact human move — from-square and to-square).
What it does
Attempts to predict the human move. The goal is for the model to learn the human way to play chess, and later to encode the position the way a human would see it, with human intuition, strength and time management.
- Strength is a dial. Pass
white_elo=1200 and it plays like a 1200. Pass white_elo=2800 and it plays like a 2800.
- No rules, no search. Legal-move understanding is emergent from training data only.
- Clock-aware. Conditioning on remaining time lets the model represent time-pressure play.
- One forward pass per move. From-square and to-square are predicted autoregressively in a single masked forward, not via tree search.
Usage
1import chess, torch
2from chessformer.model import ChessformerModel, unwrap_state_dict
3from chessformer.tokenizer import build_vocab
4from chessformer.inference import pick_move
5
6vocab = build_vocab()
7ckpt = torch.load("chessformer_v0.pt", map_location="cpu")
8model = ChessformerModel(vocab=vocab, **ckpt["cfg"]["model"])
9model.load_state_dict(unwrap_state_dict(ckpt["model"]), strict=True)
10model.eval()
11
12board = chess.Board()
13move = pick_move(model, vocab, torch.device("cpu"), board,
14 white_elo=2500, black_elo=2500,
15 white_clock_s=120.0, black_clock_s=120.0,
16 increment_s=5.0)
17print(move) # e.g. e2e4
Architecture
| Param | Value |
|---|
| Parameters | ~50M |
| d_model | 512 |
| n_heads | 8 |
| n_layers | 12 |
| FFN | SwiGLU, 4× expansion |
| Norm | RMSNorm (pre-norm) |
| Attention | Flash Attention (SDPA) |
| Max sequence length | 41 tokens |
Board pieces use additive embeddings: emb[color] + emb[piece_type] + emb[file] + emb[rank]. No positional encoding needed. Elo, clock, and increment use soft bracket interpolation.
Training
- Data: Lichess standard rated games, January 2018 (~100k games, ~6.6M positions)
- Val/Test: December 2017 (different month — no game-level leakage), Elo-balanced across 1000–2900
- Steps: 20,000
- Batch size: 1024 (2× T4 GPU, DDP)
- LR: 3e-4, cosine decay with 1000-step warmup
- Optimizer: AdamW, weight decay 0.01
Metrics (test set, Elo-balanced, 671k positions)
All numbers from the held-out test split (Lichess December 2017 — a different month from training data).
Top-1 accuracy — the model's top prediction exactly matches the human move (from-square and to-square). A random legal move scores ~3–5%.
Plausible 20% — the human move is assigned ≥ 20% probability in both the from-square and to-square distributions (the model "sees" the idea even when not picking it first).
| Metric | Value |
|---|
| Top-1 move accuracy | 44.4% |
| Plausible 20% | 62.9% |
| Puzzle solved (all moves correct) | 28.0% |
| Puzzle advancement (avg fraction solved) | 34.0% |
Accuracy is flat and best across 1100–2200 Elo, where training data is abundant. Puzzle-solving is fully emergent — the model was never trained on puzzles.
License
Apache 2.0. Training data from
database.lichess.org (CC0).
Citation
1 author = {Albert-Roulhac, Edouard},
2 title = {Chessformer: Elo-conditioned human chess imitation via transformer},
3 year = {2024},
4 url = {https://github.com/edarsem/chessformer},
5}
