A GRPO (RL) LoRA adapter on top of the
MathNano SFT model.
This run degraded the model and is kept only for the write-up — do not use it for inference.
Use the SFT adapter instead.
On a single GPU we couldn't run vLLM rollouts, so GRPO used HF generation that produced
fixed-length completions which never stopped at the answer. The verifiable reward (which reads
the last boxed value) then frequently graded correct solutions as wrong, so GRPO optimized a
corrupted signal and collapsed the policy. The lesson — GRPO is only as good as its reward's
fidelity — is the most useful outcome of the run.
