Views
No views yet
jensjepsen/danish-lm-400m-sft-v31-avg-top3. Three-way mix: gsm8k + IFEval-DA + JSON schema.| subfolder | composite | ifeval p-strict | ifeval i-strict | gsm8k | json |
|---|---|---|---|---|---|
| best1-step5500 | 2.101 | 33.4 | 49.4 | 26.6 | 93.0 |
| best2-step5125 | 2.083 | 32.5 | 48.4 | 28.5 | 96.5 |
| best3-step6250 | 2.082 | 33.0 | 48.6 | 28.0 | 98.6 |
best1-step5500grpo-mixed-v1-best1-step3375 (β=0.04, IF+gsm8k only, no JSON).| eval | v31 base | grpo-mixed-v1 | best1-step5500 |
|---|---|---|---|
| IFEval-DA p-strict | 21.2 | 29.9 | 32.7 |
| IFEval-DA p-loose | 22.0 | 30.6 | 34.0 |
| IFEval-DA i-strict | 35.2 | 45.0 | 48.8 |
| IFEval-DA i-loose | 35.8 | 45.6 | 49.9 |
| GSM8K pass@1 (n=1317) | 17.4 | 24.4 | 27.9 |
| SciQ open-Q (n=1000) | 13.5 | 14.1 | 14.1 |
| CIT-gen (n=720) | 29.9 | 28.6 | 29.2 |
| textman_summary chrF++ | 41.1 | 40.7 | 40.9 |
| textman_rewrite chrF++ | 46.5 | 47.2 | 48.2 |
| CITMC (n=720) | 48.2 | 49.6 | 47.8 |
| SciQ-MC (n=1000) | — | 59.6 | 58.7 |
| PIQA (n=100) | 53 | 59.0 | 57.0 |
| ARC-Easy chat-MC | 44.4 | 42.7 | 42.1 |
| ARC-Challenge chat-MC | 29.4 | 29.0 | 29.1 |
| ARC-Easy logprob | 40.6 | 40.5 | 41.4 |
| ARC-Challenge logprob | 27.5 | 27.3 | 26.8 |
| GPQA-Diamond chat-MC | — | 21.2 | 21.2 |
| GPQA-Diamond logprob | — | 25.8 | 23.7 |
| IFBench-DA p-strict | — | 10.3 | 8.7 |
| IFBench-DA i-strict | — | 11.3 | 9.0 |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2REPO = "jensjepsen/danish-lm-400m-grpo-mixed3-v3-beta004-top3"
3model = AutoModelForCausalLM.from_pretrained(REPO, subfolder="best1-step5500")
4tok = AutoTokenizer.from_pretrained(REPO, subfolder="best1-step5500")<|user|>{q}<|end|><|assistant|>, stop on <|end|>.