nemotron-boardgame-answer-lora-b4-safe-final is a specialist PEFT/LoRA adapter for board-game move selection and BoardGameBench-style answer generation. It continues the earlier homerquan/nemotron-boardgame-answer-lora-b4-safe-2000 adapter with preference optimization: first DPO on reviewed move pairs, then GRPO on prompt+reward maps generated from BoardGameBench states.

The headline change is better benchmark stability after DPO + GRPO: on the 22-game BoardGameBench default curriculum, this adapter reached 286.0 BRI, up from 271.9 BRI for the earlier b4-safe-2000 run, while reducing forfeits from 3 to 0.

Author: Homer Quan
Project: BoardGameBench

Training Datasets

The data used in this training lineage is published here:

homerquan/boardgamebench-answer-sft: the full answer-only SFT corpus.
homerquan/boardgamebench-answer-dpo: a reviewed DPO sample/pilot set used for the preference continuation stage.
homerquan/boardgamebench-answer-grpo: a reviewed GRPO sample/pilot prompt+reward set; the final GRPO run used a 512-example pilot slice from this release.

DPO and GRPO are intentionally published as sample/pilot datasets only, while the SFT release is the large answer-only corpus.

What This Is

This is a LoRA adapter for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16. It is not a standalone model. Load it with the compatible Nemotron 3 Nano base model through PEFT.

The adapter is narrow by design:

It is intended for compact deterministic board-game prompts.
It should be given rules, board state, side to move, and legal moves whenever possible.
It is optimized for concise move-label answers, not broad assistant behavior.

DPO + GRPO Improvement

The earlier adapter already learned the board-game answer format through SFT. The final adapter adds preference and reward optimization:

DPO: preference continuation from the most recent SFT-only adapter.
GRPO: reward-map training on BoardGameBench prompt+reward data.

In the benchmark snapshot below, DPO + GRPO preserved the single win, removed all move-format forfeits, and improved BRI.

Model	Games	Wins	Losses	Draws	Forfeits	Raw Score	Normalized	BRI
b4-safe-final, SFT + DPO + GRPO	22	1	21	0	0	5.4261/22	24.66	286.0
b4-safe-2000, earlier SFT run	22	1	21	0	3	5.37/22	24.42	271.9

Benchmark details for the final adapter are included in this repository at:

benchmarks/boardgamebench_grpo_2026-05-08.json

Per-Game Snapshot

Game	Rounds	Wins	Losses	Avg Moves	Normalized
Connect Four	4	0	4	16.5	13.75
Gomoku 19x19	2	0	2	9.5	0.92
Breakthrough 6x6	4	0	4	32.5	14.21
Dots and Boxes 3x3	4	0	4	24.0	35.00
Othello 6x6	4	0	4	33.5	30.86
Othello 8x8	2	1	1	63.5	55.96
Hex 7x7	2	0	2	37.5	26.79

These are benchmark-specific results, not a general-purpose reasoning claim. Scores can vary with prompt format, decoding settings, base model revision, hardware, and benchmark harness version.

Loading Example

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
adapter_id = "homerquan/nemotron-boardgame-answer-lora-b4-safe-final"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Example prompt shape:

text

Game: <game name>
Rules: <rules>
You are side <X/O>. Choose one legal move label exactly as written.
Legal moves: <comma-separated legal moves>
Current state:
<board>

Return the best move only.

Intended Use

Use this adapter for:

BoardGameBench evaluations
board-game move-label generation
compact deterministic game experiments
comparing SFT, DPO, and GRPO behavior on game prompts

Out of scope:

general chat or instruction following
factual QA, medical, legal, financial, or safety advice
autonomous real-world decisions
games where rules or legal moves are omitted or ambiguous

Training Details

Base model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
Adapter type: LoRA
PEFT task type: causal language modeling
Rank: 8
LoRA alpha: 16
LoRA dropout: 0.0
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Optimization stages: SFT adapter startpoint, then DPO, then GRPO
PEFT version: 0.19.1

Limitations

The adapter depends on the compatible Nemotron 3 Nano base model.
It may choose weak or illegal moves when the prompt is underspecified.
It is benchmark-specialized and should not be treated as a general-purpose assistant.
The included benchmark is a snapshot, not a proof of solved game play.

Citation

If you use this adapter in evaluations or derivative work, please cite BoardGameBench:

bibtex

@software{quan_boardgamebench_2026,
  author = {Quan, Homer},
  title = {BoardGameBench},
  year = {2026},
  url = {https://github.com/homerquan/BoardGameBench}
}

Author

Author: Homer Quan