Research artifact, not an improved model. The PPO loop runs end to end with correct telemetry,
but in this short, lightly-tuned run reward did not cleanly improve (it bounced -2 to -6 while KL to
the reference drifted 12 -> 22). The value here is the characterization of PPO's two real costs -
memory (it OOM'd on an 8 GB GPU and on an 80 GB A100 before being tuned to fit) and tuning
instability - i.e. exactly what DPO was designed to avoid. Full teardown in the repo.
