Metric	Value
Reward (mean)	0.308
Reward (max)	0.421
KL divergence (mean)	0.015
KL divergence (max)	0.035
Grad norm (mean)	3.01
Avg completion length	123.8 tokens
Training duration	1.46 hours

Reward stabilises at 0.31–0.36 vs heuristic baseline of ~0.0
KL stays controlled throughout (β=0.1, max 0.035) — no policy collapse
Completion length consistent ~122 tok — model learned JSON format fast

Eval Results

21 episodes, model as Vermillion Capital vs 6 heuristic opponents:

Metric	Value
Win rate (rank 1)	95.2% (20/21)
Survival rate	100% (21/21)
Avg episode reward	9.81 ± 1.18
Min / Max episode reward	5.62 / 11.09

Note: opponents are heuristic random agents. Next iteration uses self-play (train_grpo_selfplay.py) where all 7 companies run the trained model for a meaningful competitive benchmark.

What it does

Plays CEO of Vermillion Capital (Finance sector) in a 12-turn corporate warfare game against 6 rival companies. Reads a structured text observation each turn, outputs a JSON action:

json

{
  "private_emails": [{"to": "Nexbridge Telecom", "text": "Ally against Goldspire?"}],
  "press_release": {"claim": "Goldspire Q3 losses exposed", "marked_truthful": false},
  "action_type": "SABOTAGE",
  "action_target": "Goldspire Industries"
}

Actions: EARNINGS_CALL, SABOTAGE <target>, PARTNERSHIP <target>, PROPOSE_MERGER <target>, HOLD.

Training setup

Config	Value
Base model	`google/gemma-4-E2B-it` (Unsloth 4-bit QLoRA)
Algorithm	GRPO (TRL ≥ 0.27.0)
LoRA rank	r=16
Hardware	L40S ×1 (HF Jobs)
Steps	1181
Batch / generations	4 / 4
Learning rate	2e-6 cosine
KL β	0.1
Max completion	160 tokens

Usage

python

from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained(
    "nothr/boardroom-grpo-lora-L2-best",
    load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("nothr/boardroom-grpo-lora-L2-best")

See the environment Space for full prompt format and HTTP/WebSocket API.

Reproduce

Open the Colab notebook — free T4, ~30 min for 200 steps.

Or via HF Jobs:

bash

hf jobs uv run --flavor l40sx1 \
  --env WANDB_API_KEY=$WANDB_API_KEY \
  --env HF_TOKEN=$HF_TOKEN \
  train/train_grpo.py \
  --max-steps 500 \
  --hub-model-id yourusername/boardroom-lora

Framework versions

PEFT 0.19.1
TRL ≥ 0.27.0
Unsloth (latest)
Transformers ≥ 4.45.0