A LoRA policy from GRPO + RLVR on GSM8K (the DeepSeek-R1 / TinyZero recipe): the reward is verifiable ground truth (is the final number correct?), so there is no reward model and no value model. Part of grpo-gsm8k.

GSM8K test accuracy 0.655 -> 0.665 (+1 pt, within noise at n=200); mean reasoning length 281 -> 291. A small lift: the Instruct base is already strong on GSM8K, so a short run has little headroom (R1-Zero-style "reasoning emerges" gains need a base model and much longer training). The value is the method plus two findings - GRPO needs vLLM rollouts (~30x speedup over transformers.generate), and headroom + run-length govern the gain. Full teardown in the repo.

GRPO (TRL, use_vllm colocate), LoRA r=16 / alpha=32, group size 8, beta 0, correctness + format reward, 400 steps, 1x A100.