Trained from jensjepsen/danish-lm-400m-sft-v31-avg-top3 on a 50/25/25 mix of Danish instruction-following, gsm8k, and json-verify prompts, with DAPO-fresh dynamic resampling and TRL 1.10 + vLLM 0.19.1 colocate.

Recipe

Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3
Trainer: TRL 1.10 GRPO with DAPO-fresh dynamic sampling (GRPO_DAPO_RESAMPLE=1 GRPO_DAPO_FRESH_PROMPTS=1 GRPO_DAPO_FRESH_MATCH_TASK=1)
Data:
- IF (50%): jensjepsen/danish-if-grpo-combined-v2 — 10k prompts, 46 our + Google IFEval schema constraints, cartesian surface variation
- GSM8k (25%): rewards on #### N verification
- JSON (25%): jensjepsen/danish-json-grpo-v1 — schema-verifier reward
- Mix strategy: interleave_datasets(stopping_strategy="all_exhausted") — no subsampling
Optimizer: adamw, β1=0.9, β2=0.999, ε=1e-8, LR 1e-6 constant, 10 warmup, dr_grpo loss
Batch: 8 × 4 grad-accum × 32 generations = 1024 completions/step
Max lengths: prompt 768, completion 768
KL β: 0.004 (low — dominates 0.04 setting in prior A/B; see feedback_grpo_low_beta)
Hardware: RTX 5090 (Blackwell sm_120), fp32 master weights + bf16 autocast
Stack: TRL 1.10.0 + vLLM 0.19.1 (colocate) + transformers 4.57.6 + torch 2.10.0+cu128 + SDPA
Killed: step ~18375 for eval

Checkpoints (best-k, tracked by in-training composite reward)

subfolder	step	in-training composite	notes
`step-14375`	14375	2.177	in-training top-1
`step-14125`	14125	2.169	in-training top-2
`step-17750`	17750	2.168	in-training top-3

Composite = mean of {ifeval-da 200-row subset, gsm8k 200-row, json_verify subset} — computed on greedy generation during training via callback.

Downstream evals — all 3 GRPO ckpts vs v31 SFT base

Full test splits, chat-format (model produces letter or free text), greedy decoding. Numbers in bold mark best across the 4 columns per row.

eval / metric	v31-avg-top3 (base)	step-14375 (top-1)	step-14125 (top-2)	step-17750 (top-3)
IFEval-DA prompt-strict	21.2	36.2	39.0	41.4
IFEval-DA prompt-loose	22.0	37.1	39.5	42.3
IFEval-DA inst-strict	35.2	51.9	56.3	56.5
IFEval-DA inst-loose	35.8	52.7	57.3	57.8
IFBench-DA prompt-strict	–	10.7	10.0	10.3
IFBench-DA prompt-loose	–	16.7	16.0	15.7
IFBench-DA inst-strict	–	12.8	11.6	12.5
IFBench-DA inst-loose	–	18.0	17.4	17.7
GSM8K-DA pass@1	17.39	27.26	28.78	26.58
SciQ-DA open-Q pass@1	13.50	13.70	13.60	13.30
SciQ-DA MC-letter	–	58.50	59.70	59.40
Citizen-DA gen	29.86	29.6	29.6	28.3
Citizen-DA MC	48.19	47.1	47.1	48.9
ARC-Easy-DA chat-MC	44.40	40.24	40.82	40.82
ARC-Challenge-DA chat-MC	29.35	28.58	28.16	28.07
OpenBookQA-DA chat-MC	35.40	34.20	33.40	35.20
PIQA-DA chat-MC	53.00	49.00	50.00	57.00
Textman-DA summary chrF++	41.11	41.06	40.57	41.69
Textman-DA rewrite chrF++	46.51	44.45	44.76	44.37

Takeaways

Massive IFEval gains (+15-21pp p-strict, +17-22pp i-strict) — GRPO delivered on its training target.
Big GSM8K gain (+9-11pp pass@1) — also a training reward target.
Small regressions on ARC-Easy (-3.6 to -4.2pp), ARC-Challenge (-0.8 to -1.3pp), Citizen-gen (-0.3 to -1.6pp), Textman-rewrite (-1.8 to -2.1pp) — these tasks weren't in the training reward and drift slightly under the KL constraint.
New capability: IFBench-DA (harder AllenAI 2025 IF constraints) — v31 base doesn't have a published number but was clearly worse; step-14375 leads at 10.7% p-strict.

In-training composite is a POOR predictor of downstream ability

The composite ranks ckpts 14375 (2.177) > 14125 (2.169) > 17750 (2.168), but the fuller downstream suite ranks them inversely: 17750 wins 9 metrics, 14125 wins 5, 14375 wins 6 (biggest lead by 17750 is IFEval p-strict +5.2pp over 14375, and PIQA +8pp). The 200-row IFEval subset used during training undersamples where the ckpts differ.

Recommendation: use step-17750 for IFEval / instruction-following, step-14125 for GSM8K, and step-14375 for IFBench specifically. Or model-soup the three for a balanced ckpt.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "jensjepsen/danish-lm-400m-grpo-v34c55u8"
SUBFOLDER = "step-17750"  # or step-14125 / step-14375

tok = AutoTokenizer.from_pretrained(REPO, subfolder=SUBFOLDER)
model = AutoModelForCausalLM.from_pretrained(REPO, subfolder=SUBFOLDER, dtype="bfloat16")

prompt = "<|user|>Skriv en kort tekst om Kaj Munk.<|end|><|assistant|>"
ids = tok.encode(prompt, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256, do_sample=False,
                     eos_token_id=tok.convert_tokens_to_ids("<|end|>"))
print(tok.decode(out[0], skip_special_tokens=False))

Chat template

<|user|>{prompt}<|end|><|assistant|>{answer}<|end|> (same as v31 base)