Warm-start checkpoint for the
vGROUT
gradient-routing experiments on the
ariahw/rl-rewardhacking LeetCode
environment.
This is a 10-step GRPO checkpoint, saved the moment the first student
reward-hack appeared, with the warmup LoRA merged into the Qwen3-4B base. It sits
at the start of reward hacking: the model solves a fair fraction of problems and
has produced its first exploit of the run_tests loophole, but hacking has not
yet saturated.
The two-stage bootstrap splits capability warmup from routed RL: stage 1 warms a
student, we merge that LoRA into the base to get a frozen M0, and stage 2 runs
routed GRPO from M0 with a fresh adapter. Every arm branching from one frozen M0
makes the placebo-versus-real comparison exact.
The companion
step-20 checkpoint
is too saturated for a warm start (it deploy-hacks ~0.84), so this earlier
first-hack point is the default warm start.
