You can deploy this model and start earning money today!

jensjepsen/
danish-lm-400m-grpo-mixed3-combinedv4-topk

Name: danish-lm-400m-grpo-mixed3-combinedv4-topk
Availability: InStock

safetensors

danish

grpo

instruction-following

jensjepsen/danish-if-grpo-combined-v4

jensjepsen/danish-lm-400m-sft-v31-avg-top3

finetune

mit

Views

No views yet

danish-lm-400m-grpo-mixed3-combinedv4-topk

Top-3 checkpoints (by tracked best-k score) from a 3-way GRPO run on jensjepsen/danish-lm-400m-sft-v31-avg-top3, mixing instruction-following, gsm8k, and JSON schema reward at 1:1:1. Paused at step 18455/89709 (~21%).

Subfolders (best-k ranked):

step-16625 — top-1, best-k score 2.360
step-17875 — top-2, best-k score 2.345
step-17500 — top-3, best-k score 2.342

Recipe

Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3
KL ref: same (jensjepsen/danish-lm-400m-sft-v31-avg-top3)
IF source: jensjepsen/danish-if-grpo-combined-v4 (10k rows)
Mix: interleaved with gsm8k + JSON at --gsm-frac 0.333 --json-frac 0.333
Rollouts: bs=8 × grad-accum=4 × ngen=32 = 1024 rollouts/step
Optimizer: dr_grpo, LR 1e-6, β 0.004, warmup 10
DAPO: fresh-same-task resampling (GRPO_DAPO_RESAMPLE=1 GRPO_DAPO_FRESH_PROMPTS=1 GRPO_DAPO_FRESH_MATCH_TASK=1)
vLLM: colocate, --vllm-gpu-memory-utilization 0.40, dr_grpo loss
Selection: best-k on running composite (ifeval-da inst-strict + gsm8k pass@1 + json mean-reward, every 125 steps)

Wandb

https://wandb.ai/jepsen/danish-lm-grpo/runs/zo0m286i (run id zo0m286i)

Results

Full downstream suite, 0-shot greedy generation (temp=0, rp=1.1, max_new_tokens=512-768 per eval script). Aggregate row = mean of matching-benchmark rows in the v31 SFT report where computable.

Eval	Metric	v31 SFT (base)	step-16625 (top-1)	step-17875 (top-2)	step-17500 (top-3)
ifeval-da	prompt-strict	21.2 (avg-top3)	41.7	41.6	39.1
	inst-strict	35.2	58.2	58.0	56.3
ifbench-da	prompt-strict	—	11.7	9.3	10.0
	inst-strict	—	13.1	11.3	11.6
gsm8k (pass@1)	acc	17.39	28.93	29.76	27.79
json (mean_reward)	—	—	1.041	1.052	1.052
sciq-gen (pass@1)	acc	13.50	13.60	13.50	14.20
sciq-mc	acc	—	59.60	59.30	59.70
cit-gen	acc	29.86	27.90	27.40	27.50
cit-mc	acc	48.19	48.50	48.90	49.00
arc-easy chat-mc	acc	44.40	40.78	40.53	40.53
arc-challenge chat-mc	acc	29.35	27.56	27.73	28.07
openbookqa chat-mc	acc	35.40	36.60	35.80	36.60
piqa chat-mc	acc	53.00	53.00	53.00	53.00
textman-summary	ChrF++	41.11	40.62	41.10	41.13
textman-rewrite	ChrF++	46.51	46.43	46.38	46.71

v31 SFT numbers from the avg-top3 SFT README.

Where GRPO helped (vs v31 SFT base)

ifeval-da prompt-strict: 21.2 → 41.7 (+20.5pp) — near-doubled
ifeval-da inst-strict: 35.2 → 58.2 (+23pp)
gsm8k pass@1: 17.39 → 29.76 (top-2) (+12.4pp)

Where GRPO neutral

sciq-gen / cit-mc / openbookqa / piqa / textman: ±1pp vs SFT

Where GRPO regressed

cit-gen: 29.86 → 27.90 (top-1) (-2pp)
arc-easy chat-mc: 44.40 → 40.78 (top-1) (-3.6pp)
arc-challenge chat-mc: 29.35 → 27.56 (top-1) (-1.8pp)

Which checkpoint to use

step-16625 (top-1): best on format-heavy tasks (ifeval, ifbench, cit-gen, arc-easy). Pick for instruction-following.
step-17500 (top-3): best on reasoning/reading (arc-challenge, sciq, cit-mc, textman). Pick for QA/reading.
step-17875 (top-2): balanced — best gsm8k, mid on both fronts.

Deltas are <1pp on most metrics — the three are close.

Chat template

<|user|>{prompt}<|end|><|assistant|>{answer}<|end|>

jensjepsen/danish-lm-400m-grpo-mixed3-combinedv4-topk