LoRA adapters accompanying "The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning." Paper (arXiv) · Code (GitHub) · Zenodo · Training Data (Kaggle)

Summary

This repository contains six LoRA adapters, all fine-tuned from Qwen/Qwen3-14B, tracing a chess-playing model through supervised fine-tuning, two explicit chain-of-thought reinforcement learning runs, a latent-reasoning curriculum, and two reinforcement learning runs on top of that latent curriculum. Together they form the full experimental arc behind the paper's central finding: reinforcement learning improves legal-move rate and eliminates checkmate confabulation, but causal intervention shows the model does not depend on the content of its latent "thoughts" to achieve either gain. The improvement is encoded in the adapter weights, not in inference-time computation over the thought vectors.

Checkpoints

Subfolder	Stage	Legal move rate	Accuracy	False checkmates	Notes
`sft-qwen3-14b-chess`	Imitation baseline	38%	9%	28	Supervised fine-tuning only, no reinforcement learning.
`rung1-grpo-v1-explicit-chess`	Explicit CoT + RL (sparse reward)	52%	8%	0	Superseded. Sparse reward let legality crowd out move quality; Stockfish centipawn-loss comparison shows degraded play relative to the SFT baseline despite the legal-move gain. Retained for provenance.
`rung1-grpo-v2-explicit-chess`	Explicit CoT + RL (gated dense reward)	52%	10%	0	Fixes the v1 quality regression with a gated dense reward. This is the explicit-reasoning result reported in the paper.
`rung2-stage2-latent-chess`	Latent curriculum, no RL	48%	10%	19	The model's reasoning has been compressed into latent thought vectors, but no reinforcement learning has been applied yet.
`rung3-latent-grpo-chess`	Latent curriculum + RL	61%	9%	0	The headline result. Highest legal-move rate across every checkpoint, with false checkmates fully eliminated.
`rung3-gumbel-chess`	Latent curriculum + RL (Gumbel-Softmax variant)	~50%	—	0	A second, independent RL run using Gumbel-Softmax straight-through estimation instead of the primary Rung 3 method. Confirms the same qualitative pattern with a smaller absolute gain; included as a robustness check, not a replacement for `rung3-latent-grpo-chess`.

Format compliance is 100% for every checkpoint except where the causal interventions described in the paper deliberately push the model out of distribution.

Usage

Every checkpoint is a LoRA adapter, not a standalone model. Loading one requires the base model and the peft library. Select a checkpoint using subfolder.

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE_MODEL = "unsloth/qwen3-14b-bnb-4bit"   # 4-bit NF4 base used during training and evaluation
REPO_ID = "unpairedelectron07/ChessGRPO-The-Weight-of-Silence"
CHECKPOINT = "rung3-latent-grpo-chess"       # swap for any subfolder in the table above

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

base = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, subfolder=CHECKPOINT)
model = PeftModel.from_pretrained(base, REPO_ID, subfolder=CHECKPOINT)
model.eval()

Prompt Format

The explicit-reasoning checkpoints (sft-qwen3-14b-chess, rung1-grpo-v1-explicit-chess, rung1-grpo-v2-explicit-chess) expect written reasoning inside <thinking> tags, structured under [KING SAFETY], [CHECKS], [CAPTURES & TRADES], [THREATS], and [IMPROVEMENT] headers, followed by a move in UCI format inside <output> tags.

The latent checkpoints (rung2-stage2-latent-chess, rung3-latent-grpo-chess, rung3-gumbel-chess) replace written reasoning with a fixed number of continuous "thought" vectors generated between two special tokens, rather than text. Reproducing this behavior requires the thought-generation loop described in the paper and implemented in the GitHub repository's interpretability/ and rung3_latent_grpo/ scripts; a chat template alone is not sufficient for these checkpoints.

Why the Latent Checkpoints Do Not Behave the Way They Might Appear To

Reading the model card alone might suggest the latent checkpoints "think silently" in a way analogous to explicit chain-of-thought. The paper's central finding is that this is not the case. A six-condition causal intervention suite, applied to both rung2-stage2-latent-chess and rung3-latent-grpo-chess, shows that substituting, adding noise to, or entirely ablating the latent thought vectors leaves performance largely unchanged. Only replacing them with all-zero vectors causes a collapse, which reflects an out-of-distribution failure rather than evidence that the thought content was doing meaningful work. Full methodology and statistical testing are in the paper.

Training Data

Training data (12,002 rows of FEN positions, best moves, and structured reasoning) is available separately on Kaggle. Link forthcoming.

License

These adapters are released under the Apache 2.0 license, matching the base model. Use of these adapters requires compliance with the Qwen3-14B license terms for the base model itself.

Citation

bibtex

@misc{kshirsagar2026weightofsilence,
  title         = {The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning},
  author        = {Kshirsagar, Ishan S.},
  year          = {2026},
  eprint        = {2607.20952},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2607.20952}
}

About

Ishan S. Kshirsagar, independent researcher. Paper: arxiv.org/abs/2607.20952