The trained LoRA adapters from the fairness-GRPO v2 campaign: can outcome-aligned RL (GRPO on an engine-computed, text-blind log-Nash reward) make an LLM negotiator distribute surplus more fairly on held-out games — not merely close more deals — and generalize to settings where the numbers are not machine-readable?

The evaluation record for these adapters is the dataset siddharthmb/2026.RA.Fairness-GRPO-v2 (contrasts, telemetry, verdicts). This repo is the weights those numbers were computed from. Full methodology + the machine-derived final tables: research note experiments/rational_agents/research-notes/0028-fairness-grpo-v2.md; narrative hub: https://rational-agents-runs.sidmb.com/grpo-v2-lam1-two-attractors/.

What this is

Each folder <arm>_checkpoint-<STEP>/ is a complete PEFT adapter (r32/α64, LoRA on q/k/v/o/gate/up/down) over Qwen/Qwen3-8B, thinking off, saved at global training step STEP. STEP is continuous within an arm across its resume segments (see "Training provenance"). Three arms trace the λ-mixture, the swept variable — seat i trains on R_i(λ) = (1−λ)·g(z_i) + λ·R_table, where g is the smoothed clipped log-Nash utility (g_floor = −17.5) and R_table is the mean over seats:

arm	λ	reward	checkpoints (global step)
`lam1`	1.0	pure table welfare	5, 10, 15, 25, 30, 35, 40, 45, 50
`lam05`	0.5	mixture	5, 10, 15, 20, 25, 30, 35, 40, 45, 50
`lam0`	0.0	pure self-interest (selfish control, shaping off)	5, 10, 15

Headline result: the north-star answer is no

Every interpretable rung of every arm fails the preregistered POSITIVE gate. Held-out primary deal-rate degrades at every λ; prose-transfer fairness never resolves favorable. The verdict is a clean, fully-measured negative for this reward family — for negotiation fairness specifically, an algorithmic agent + a parser dominates the RL'd LLM.

Held-out primary deal-rate delta vs untrained Qwen3-8B (paired, game-cluster bootstrapped; the eval bank is instances_grpo_eval_v1):

step	5	10	15	20	25	30	35	40	45	50
λ=1.0	−0.052	−0.143	−0.147	—	−0.226	—	—	−0.453	−0.641	−0.746
λ=0.5	−0.058	−0.046	−0.058	−0.151	−0.267	−0.240	−0.445†	−0.173	−0.271	−0.369
λ=0	−0.028	−0.075	(partial‡)

(closure-conditional fairness metrics are stamped VOID at any rung whose deal rate falls >0.10 below base — a viability screen, Amendment 5(a). † rung 35 sits immediately after a fresh-Adam restart and is anomalously deep. ‡ λ=0 rung 15 stopped for cost, 346 eps, reported partial.)

What the frontier means: closure decay scales with how much a seat is paid on the table rather than on itself. λ=1.0 collapses monotonically to a policy that will not sign at all (prose closes 0.002, ultimatum closes nothing). λ=0.5 buys robustness of closure (half the depth, oscillation not monotone collapse) but never fairness. λ=0 (walk fully priced to self) shows no closure collapse through rung 10 — isolating table-welfare payment as the collapse mechanism, not GRPO-on-this-task.

The sharpest single finding — boundary extraction (from lam1_checkpoint-25): on the ultimatum family the policy learned a 100/0 split accepted every time, placing the responder exactly on its walk-away point (z<ε). Every closure-conditional guard reads this as perfect; only a share/dispersion term (max_share) sees it. Standing program rule: any below-threshold gate carries a share term beside the deal-rate floor.

How to use an adapter

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype=torch.bfloat16, device_map="cuda")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "siddharthmb/2026.RA.Fairness-GRPO-v2-Adapters", subfolder="lam1_checkpoint-25")

Reproduce the training

Per arm (via experiments/rational_agents/grpo_v2/train.py), Qwen3-8B LoRA r32/α64, thinking off, g_floor=−17.5, GRPO groups=22 × k=8, frozen population-opponent zoo {bayes-rational, passive-gate, greedy-anchor}:

bash

# λ=1.0 (arm lam1): steps 5-25 at groups=22; resumed from checkpoint-25 for the dense tail 30-50.
python -m grpo_v2.train --lam 1.0 --steps 50 --groups 22 --k 8 --bank instances_grpo_v2_train \
    --checkpoint-steps 5 10 15 25 30 35 40 45 50 --g-floor -17.5 \
    --wandb-run-name grpo_v2_lam1 --wandb-group fairness-grpo-v2
# λ=0.5 (arm lam05): --lam 0.5, same knobs; groups reduced 22→14→12 across resume segments (see provenance).
# λ=0  (arm lam0):  --lam 0.0 --shaping-weight 0 --steps 15 --checkpoint-steps 5 10 15  (selfish control, note 0050).

Training provenance (width changes across resume segments)

Both larger arms were trained across compute migrations (RunPod balance exhaustion → Lambda), so the rollout width (groups) changed mid-arm. This is recorded because the GRPO group-relative baseline normalizes per-instance, so width does not confound the contrasts, but consumers comparing rungs should know it (0028 Amendments 9–11):

λ=1.0: steps 5–25 and the resume 30–50 both at groups=22 (B200). One fresh-Adam discontinuity at the step-25→30 resume boundary.
λ=0.5: steps 1–15 at groups=22 (RunPod B200) → 16–31 at 14 → 32–50 at 12 (Lambda H100, 80 GB fit). Fresh Adam at each resume; a completion-token-alignment guard change (drop small-drift rows) landed mid-arm.
λ=0: 15 steps single segment, shaping off.

Weights & Biases

Project wandb.ai/siddharth-stanford/rational_agents_fairness_grpo, group fairness-grpo-v2. Runs: grpo_v2_lam1 (npz11gav) → grpo_v2_lam1_resume25; grpo_v2_lam05 → grpo_v2_lam05_resume15 → grpo_v2_lam05_resume30b; grpo_v2_lam0.

Cluster paths

Adapters (source): /nlp/scr/siddharth/ii_mats/rational_agents/grpo_v2/{lam1,lam1_resume25,lam05,lam05_resume15,lam05_resume30b,lam0}/checkpoint-*
Eval verdicts (JSON): experiments/rational_agents/results/fairness_grpo_v2/eval_*.json
Eval run dirs: /nlp/scr/siddharth/ii_mats/rational_agents/grpov2eval_*
Rollout transcripts: .../grpo_v2/*/transcripts/; training telemetry .../*/steps.jsonl
Log: experiments/rational_agents/self_benefit/LOG.md; note: research-notes/0028-fairness-grpo-v2.md; λ=0 control: research-notes/0050-lam0-selfish-control-prereg.md

Relation

Supersedes the v1 pilot adapters (2026.RA.Fairness-GRPO-lam1, -lam0; dataset 2026.RA.Fairness-GRPO), whose "discipline without distribution" negative (note 0023) is v2's design input. Base model: Qwen/Qwen3-8B.