BF16 inference checkpoints for allenai/Olmo-3-7B-Think-DPO, trained with GRPO using only the instruction-following constraint reward. Anchor/PPL rewards are disabled.

Each checkpoint is a complete Transformers model in its own subfolder:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "just1nseo/olmo3-7b-think-if-rlvr-constraint-only-pp0-8k"
subfolder = "global_step_364"

tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    subfolder=subfolder,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

OLMo 3 requires a Transformers version with native olmo3 support (the training runtime used Transformers 4.57.1).

Checkpoints

Subfolder	Completed epoch
`global_step_91`	1
`global_step_182`	2
`global_step_273`	3
`global_step_364`	4

Training configuration

Setting	Value
Experiment	`olmo3_7b_grpo_think_constraint_only_b1024_c1_pp0_8k`
Base and reference model	`allenai/Olmo-3-7B-Think-DPO`
Training epochs / total steps	4 / 364
Train batch size	1024
Rollouts per prompt	8
Maximum prompt / response length	2048 / 8192
Rollout sampling	temperature 0.6, top-p 0.95, presence penalty 0.0
Actor learning rate	`1e-6`
KL loss	low-variance KL, coefficient `0.001`
IF reward	fraction of satisfied IFEval constraints
Auxiliary PPL / anchor rewards	Disabled (both coefficients `0.0`; anchor precompute/policy scoring off)
Reasoning format	OLMo Think template; a closing `</think>` is required for IF reward

The full FSDP trainer checkpoints, including optimizer and data-loader state, are retained separately. This repository contains merged BF16 model weights intended for inference and evaluation.