Views
No views yet
| Dataset | Base | Tuned (mean ± std) | Δ (pp, rel %) |
|---|---|---|---|
| gsm8k | 81.2 | 49.5 ± 28.6 | -31.7 pp, -39% |
| arc_challenge | 85.2 | 68.2 ± 3.1 | -17.0 pp, -20% |
| arc_easy | 97.7 | 86.7 ± 3.6 | -11.0 pp, -11% |
| commonsenseqa | 69.8 | 60.2 ± 5.9 | -9.7 pp, -14% |
| openbookqa | 82.3 | 56.8 ± 2.0 | -25.5 pp, -31% |
| qasc | 76.5 | 61.7 ± 3.6 | -14.8 pp, -19% |
| sciq | 93.8 | 83.8 ± 4.3 | -10.0 pp, -11% |
| mmlu_pro(OOD) | 33.8 | 21.3 ± 2.0 | -12.5 pp, -37% |
| mmlu_redux(OOD) | 52.5 | 45.2 ± 2.8 | -7.3 pp, -14% |
| gpqa_diamond(OOD) | 7.6 | 33.2 ± 0.8 | +25.6 pp, +338% |
| InD Average | 83.8 | 66.7 ± 3.9 | -17.1 pp, -20% |
| OOD | 31.4 | 33.2 ± 0.9 | +1.8 pp, +6% |
| ALL | 68.1 | 56.7 ± 2.9 | -11.4 pp, -17% |
pp) and the relative percent change (tuned − base) / base × 100 (rel %, shown as n/a when base accuracy is 0).| Dataset | Base | Tuned (mean ± std) | Reduction % |
|---|---|---|---|
| gsm8k | 4450 | 565 ± 397 | -87% |
| arc_challenge | 3157 | 89 ± 4 | -97% |
| arc_easy | 1871 | 85 ± 3 | -95% |
| commonsenseqa | 3949 | 72 ± 1 | -98% |
| openbookqa | 3378 | 78 ± 1 | -98% |
| qasc | 3932 | 86 ± 4 | -98% |
| sciq | 1944 | 79 ± 2 | -96% |
| mmlu_pro(OOD) | 6582 | 126 ± 8 | -98% |
| mmlu_redux(OOD) | 5589 | 101 ± 3 | -98% |
| gpqa_diamond(OOD) | 8001 | 121 ± 6 | -98% |
| InD Average | 3240 | 151 ± 55 | -95% |
| OOD | 6720 | 116 ± 2 | -98% |
| ALL | 4281 | 140 ± 39 | -97% |
+, means the tuned model generates more tokens).