Exploration via the sampling distribution. Trained at rollout temperature 1.2 but VALIDATED at 1.0 like every other arm, so its curve stays comparable.

Training data and format reward


dataset	ScienceQA (`scienceqa`)
epochs / steps	25 / 1250
batch / rollouts	128 prompts, K=6
learning rate	1e-6 constant
KL (in-reward)	0.01
max prompt / response	512 / 1024 tokens
format reward	0.03, constant, no decay
seed	42

The training prompt requests the numbered-step scaffold and a final \boxed{X}, matching the format reward. It does not carry the "IMPORTANT rules" emphasis block used by the AoPS suite, so the wording (not the requested output format) differs from that suite.

Arm-specific: entropy_coeff=0.0, clip=0.2/0.2, rollout temperature=1.2.

Do not apply a chat template

Trained on raw prompt text. verl's RLHFDataset has apply_chat_template=False and it was never enabled. Applying Qwen2.5-Math's chat template at inference creates a train/eval mismatch measured at roughly 19 points of pass@1.

python

from vllm import LLM, SamplingParams
llm = LLM(model="sandeep123/sqa-grpo-temp12-step900", dtype="bfloat16", max_model_len=1536)
params = SamplingParams(n=6, temperature=1.0, top_p=1.0, top_k=-1, max_tokens=1024)
out = llm.generate([prompt_text], sampling_params=params)   # raw string, not llm.chat()

Validation metrics at this checkpoint

metric	value
pass@1	0.8359
pass@6	0.9297
step	900

Answer extraction (pre-registered). An answer is the content of the final \boxed{}; if absent, the last standalone A-E token. Responses with no extractable answer are scored incorrect, and all K rollouts stay in the denominator.

Validation uses 256 held-out prompts, K=6, temperature 1.0, seed 42 -- pinned in code so every arm, including the temperature-1.2 arm, is scored under identical decoding.

Checkpoint selection

Quality-optimal and diversity-optimal checkpoints differ substantially, so both the best-pass@1 and best-pass@6 checkpoints are published for every arm. Diversity results are therefore never reported from a checkpoint chosen purely for accuracy.