This repository is not a single model. It is the complete checkpoint family of one full-parameter GRPO run on Qwen/Qwen3-4B: the policy exported every 10 optimizer steps, from step 10 to step 250, as 25 self-contained Hugging Face model directories.

The point of publishing all 25 is the trajectory, not the endpoint. Work on speculative decoding, drafter drift, reward hacking, and RL training dynamics generally needs to compare a policy against itself at nearby points in training, and that is exactly what is normally thrown away when only the final checkpoint is released.

Layout

step_10/ step_20/ step_30/ ... step_250/

Each step_N/ is a standard HF model directory (config.json, model-0000*-of-00002.safetensors, model.safetensors.index.json, tokenizer files) and loads directly:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
    subfolder="step_250",
    dtype="bfloat16",
)
tok = AutoTokenizer.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts", subfolder="step_250"
)

Or fetch a single step without pulling the whole 188 GB:

hf download TIE-Pilot/qwen3-4b-grpo-dapo-ckpts --include "step_250/*" --local-dir ./ckpt

Training setup

Trained with verl, full-parameter FSDP, vLLM rollouts.


Base model	`Qwen/Qwen3-4B`
Algorithm	GRPO (`adv_estimator=grpo`)
Dataset	DAPO-Math
KL	`use_kl_in_reward=False`, `use_kl_loss=True`, `kl_loss_coef=0.001`, `low_var_kl`
Learning rate	1e-6
Train batch / mini-batch	64 / 32 (16 / 8 per GPU × 4 GPUs)
Rollouts per prompt	8, temperature 0.9
Max prompt / response	1024 / 4096 tokens
Epochs	8
Save frequency	every 10 steps
Precision	bfloat16 weights

Checkpoints were reconstructed from verl's sharded FSDP output with verl.model_merger merge --backend fsdp.

Caveats

No per-step evaluation is included. These are raw policy exports; reward curves and benchmark scores were not published alongside them.
Step 250 is the final checkpoint of the run, not a checkpoint selected for best validation performance.
This is a research byproduct from a speculative-decoding project. It is a standard GRPO run and is not intended to be a strong math model — its value is the density of the trajectory.

7B sibling family: TIE-Pilot/qwen25-7b-base-grpo-dapo-ckpts

TIE-Pilot/qwen3-4b-grpo-dapo-ckpts

Qwen3-4B GRPO/DAPO-Math — dense checkpoint family (25 steps)

Layout

Training setup

Caveats

Related

TIE-Pilot/
qwen3-4b-grpo-dapo-ckpts