You can deploy this model and start earning money today!

nitishpandey04/
alphago-zero-5x5

Name: alphago-zero-5x5
Availability: InStock

reinforcement-learning

mit

Views

No views yet

AlphaGo Zero–style Go Agent (5×5)

A small policy–value network trained entirely from self-play (no human games, no supervised pretraining) using the AlphaGo Zero algorithm, on a 5×5 Go board. The agent started knowing only the rules and improved by playing against its own best version, with Monte-Carlo Tree Search (MCTS) providing a search-improved policy target each iteration.

📦 Code, training loop, and how-to-run: https://github.com/nitishpandey04/alphago

This repo hosts only the trained weights (best.pt) and this card. Everything needed to use them — play against the agent, watch self-play, or reproduce the handicap study — lives in the GitHub repository above.

Model details


Architecture	Residual conv trunk + policy & value heads (AlphaGo Zero, scaled down)
Parameters	314,466
Input	`(4, 5, 5)` planes: own stones, opponent stones, ko point, color-to-move (side-to-move relative)
Trunk	4 residual blocks, 64 channels, BatchNorm + ReLU
Policy head	1×1 conv (2 ch) → linear → logits over `5*5 + 1 = 26` actions (incl. PASS)
Value head	1×1 conv (8 ch) → linear → ReLU → linear → `tanh` ∈ [−1, 1]
Board	5×5, komi 3.5, simple ko, area scoring
Training	100 iterations of self-play → train → arena, batched MCTS on a single RTX 5060 Ti
Framework	PyTorch (CUDA 12.8 wheels for Blackwell sm_120)

The training loss fell from ~2.30 → ~0.76 (policy ~2.0 → ~0.60, value ~0.30 → ~0.16) over the run, with the candidate promoted past the arena gate 8 times.

How to use

bash

# 1. Get the code
git clone https://github.com/nitishpandey04/alphago
cd alphago
uv sync                       # PyTorch (cu128), numpy, matplotlib, tensorboard

# 2. Get these weights
hf download nitishpandey04/alphago-zero-5x5 best.pt --local-dir checkpoints

# 3a. Play against it (you are Black; enter moves like C3, B5, or 'pass')
uv run python -m scripts.play_human --checkpoint checkpoints/best.pt

# 3b. Watch it play itself
uv run python -m scripts.watch_game --black checkpoints/best.pt --white checkpoints/best.pt

# 3c. Handicap one side from the SAME weights (fewer sims = weaker search)
uv run python -m scripts.watch_game --black-sims 32 --white-sims 64 | tail -1

Minimal inference (load + one forward pass)

python

import torch
from go_agent.model import PolicyValueNet
from go_agent.game import Board
from go_agent.encoding import encode

net = PolicyValueNet(board_size=5, in_channels=4, num_res_blocks=4,
                     channels=64, value_channels=8, hidden_size=64)
net.load_state_dict(torch.load("checkpoints/best.pt", map_location="cpu"))
net.eval()

board = Board(5)                              # empty 5×5, Black to move
x = torch.from_numpy(encode(board)[None])     # (1, 4, 5, 5)
policy_logits, value = net(x)                 # logits over 26 actions, value ∈ [−1, 1]

For real play, wrap the network in MCTS — see go_agent/mcts.py and the scripts in the repo (the raw policy alone is much weaker than policy + search).

The handicap study (the fun part)

On 5×5, Black's first-move advantage is enormous — at equal search the agent wins as Black ~97% of self-play games. So: how badly must you handicap Black's search before White's deeper thinking wins? Because both sides can run the same checkpoint at different MCTS simulation counts, you can measure it.

Result (48 games/point). Fixing White at 64 sims, Black breaks even at only ~2 simulations — a ~32× search deficit — and only loses the majority when reduced to 1 sim (raw policy, no lookahead). Doubling White to 128 sims exposes a neat split between two metrics:

Win-rate crossover barely moves (~2 → ~3 sims): the first-move advantage is so structural that a couple of simulations win more often than not regardless of how hard White searches.
Score-margin even point shifts ~3× (~6 → ~20 sims): deeper White search makes White lose by less, not win more often.

Full graphs (both White budgets + a side-by-side), methodology, and the reproducible handicap_study.py are in the GitHub README.

Limitations

5×5 only — these weights are board-size specific. Scaling to 7×7/9×9 requires retraining (config-only change in the repo).
Simple ko (no positional superko); area scoring without a dead-stone removal pass. Fine for self-play; not fully tournament-legal.
The raw network is weak without MCTS — always pair it with search for play.

License & attribution

MIT. An independent, educational reimplementation of the AlphaGo Zero algorithm; not affiliated with or endorsed by DeepMind.