The model was trained on a subset of the Lichess 2025 dataset, focusing on learning valid chess move sequences. The architecture was carefully tuned to stay within the 1M parameter constraint while maintaining reasonable performance.

Usage

Parameters: 948,352 (0.95M)
Vocabulary size: 85
Embedding dimension: 128
Number of layers: 6
Attention heads: 4
Feed-forward dimension: 320
Context length: 256
Dropout: 0.105

python

from transformers import AutoModelForCausalLM
from src.tokenizer import ChessTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "LLM-course/chess-ooooooooo-5",
    trust_remote_code=True
)
tokenizer = ChessTokenizer.from_pretrained(
    "LLM-course/chess-ooooooooo-5",
    trust_remote_code=True
)

# Generate moves
input_text = "[BOS] WPe2e4"
input_ids = tokenizer.encode(input_text)
outputs = model.generate(input_ids, max_length=50)
predicted_moves = tokenizer.decode(outputs[0])

Submission

Submitted by etienneLefranc for the LLM Course Chess Challenge. Version 5 of chess-ooooooooo.