Alpha release. A chess LLM trained from scratch on the Lichess dataset using the Qwen3-0.6B architecture and a custom UCI move tokenizer. Expect rough edges — move quality, strategy, and robustness are all unvalidated beyond basic smoke tests.

Source: https://github.com/jspaulsen/halluci-mate

Model details

Architecture: Qwen3 (Qwen3ForCausalLM), ~0.6B parameters
- 28 layers, hidden size 1024, 16 attention heads (8 KV heads), intermediate size 3072
- bfloat16, tied word embeddings, RoPE θ = 1,000,000
Vocabulary: 1,974 tokens — 6 special tokens (<PAD>, <UNK>, <EOS>, <WHITE>, <BLACK>, <DRAW>) + ~1,792 geometric UCI moves + 176 promotion moves
Context: 32,768 tokens
Checkpoint: runs-v1/marvelous-deer-608/checkpoint-9660

Tokenizer

The tokenizer is custom and is not loadable via AutoTokenizer.from_pretrained. It is defined in src/halluci_mate/chess_tokenizer.py in the source repo. Install the package and use ChessTokenizer() directly.

Inputs are conditioned on the side-to-move winning: each game is prefixed with <WHITE> or <BLACK> (or <DRAW>), followed by the sequence of UCI moves.

Usage

python

import chess
import torch
from transformers import AutoModelForCausalLM

from halluci_mate.chess_tokenizer import ChessTokenizer
from halluci_mate.game.game import Game
from halluci_mate.inference import ChessInferenceEngine

engine = ChessInferenceEngine.from_checkpoint(
    "jspaulsen/halluci-mate-v1a",
    constrained=True,   # mask logits to legal moves
    temperature=0.0,    # greedy
)

game = Game(board=chess.Board(), condition="<WHITE>")
move = engine.predict(game)
print(move.uci())

Constrained decoding masks the logits to the set of legal UCI moves in the current position, which eliminates illegal-move hallucinations at the cost of potentially hiding model weaknesses. Unconstrained sampling (constrained=False) will occasionally produce illegal tokens — this is expected for an alpha.

Training

Data: Lichess games, filtered to Normal termination, SAN parsed to UCI with python-chess
Model initialized from config (no pretrained weights) via AutoModelForCausalLM.from_config
Training script: scripts/train.py in the source repo

Hyperparameters, dataset size, and eval metrics are not finalized — see TODO.md in the source repo.

Limitations

Alpha quality; move strength has not been benchmarked against a rated engine
Constrained decoding is recommended for any real use — the raw model may emit illegal move tokens
Trained on human games, so idiosyncrasies and blunders at lower ratings are reflected in behavior
No support for analyzing positions from arbitrary FENs beyond what Game constructs

License

MIT. See the source repo for details.