You can deploy this model and start earning money today!

Thunderous77/
grpo

Name: grpo
Availability: InStock

Views

No views yet

GRPO / GCPO / DAPO / AGRO checkpoints

This public repository contains 15 merged bfloat16 Hugging Face checkpoints from 9 reinforcement-learning experiments based on Qwen/Qwen3-1.7B-Base (base revision ea980cb0a6c2ae4b936e82123acc929f1cec04c1).

Layout

Experiment	Repository subfolder	Steps
AGRO SeqSum, beta=0.001	`agro-seqsum-beta0.001/step-{N}`	100
GCPO Exp SeqMean, beta=0.001	`gcpo-exp-seqmean-beta0.001/step-{N}`	360, 500
GCPO Exp SeqMean, beta=0.01	`gcpo-exp-seqmean-beta0.01/step-{N}`	340, 500
GCPO Exp SeqMean, beta=0.1	`gcpo-exp-seqmean-beta0.1/step-{N}`	500
GCPO Exp SeqMean, beta=1	`gcpo-exp-seqmean-beta1/step-{N}`	500
GCPO Exp SeqMean, beta=1, plain shuffle	`gcpo-exp-seqmean-beta1-plain-shuffle/step-{N}`	440, 500
GRPO, KL=0.001	`grpo-kl0.001/step-{N}`	420, 500
Matched DAPO	`matched-dapo/step-{N}`	220, 300
Matched DAPO + Token-TIS	`matched-dapo-tis/step-{N}`	220, 300

Each subfolder is a standalone Transformers model containing merged model.safetensors, model configuration, and tokenizer files.

Checkpoint pruning (2026-08-25)

Intermediate checkpoints were pruned; each experiment now keeps its peak checkpoint (selected by the training-time MATH500 validation curve) and/or its last checkpoint. The per-step MATH500 validation history for every experiment remains fully recorded in W&B.

Loading

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Thunderous77/grpo"
subfolder = "gcpo-exp-seqmean-beta1-plain-shuffle/step-500"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder=subfolder,
    dtype="auto",
    device_map="auto",
)

Important scope

These uploads are standard merged model weights intended for inference, evaluation, or further initialization. Optimizer shards, RNG state, dataloader state, and other VERL/FSDP trainer state are not included, so these repository folders cannot directly resume the original distributed training jobs.

Before local cleanup, every source checkpoint was checked for complete 8-way model/optimizer/extra-state shards. Every merged remote model.safetensors was then verified against its local SHA-256 digest.