This is a research checkpoint derived from Qwen/Qwen3-30B-A3B-Base with group-relative policy optimization on mathematical reasoning data. It was exported at training step 800.

The training run used three stabilization methods:

length-aware adjustment of long negative-advantage responses;
down-weighting of locally high-mismatch response segments;
a small one-sided log-probability alignment loss.

The checkpoint is provided as standard BF16 Hugging Face safetensors. The stabilization methods affect training only and add no custom inference code.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "shawnzzzzz/Qwen3-30B-A3B-GRPO-3Loss-Step800"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

Use transformers>=4.51.0 for Qwen3-MoE support.

Notes

This is an intermediate research checkpoint, not a production release. Users should independently evaluate correctness, safety, and suitability for their deployment setting. The base-model license applies.