Views
No views yet
| Dataset | Base | Tuned (mean ± std) | Δ (pp, rel %) |
|---|---|---|---|
| gsm8k | 81.2 | 6.5 ± 0.5 | -74.7 pp, -92% |
| arc_challenge | 85.2 | 76.2 ± 2.6 | -9.0 pp, -11% |
| arc_easy | 97.7 | 91.7 ± 1.6 | -6.0 pp, -6% |
| commonsenseqa | 69.8 | 69.2 ± 0.8 | -0.7 pp, -1% |
| openbookqa | 82.3 | 66.7 ± 1.8 | -15.7 pp, -19% |
| qasc | 76.5 | 66.0 ± 4.4 | -10.5 pp, -14% |
| sciq | 93.8 | 91.3 ± 0.6 | -2.5 pp, -3% |
| mmlu_pro(OOD) | 33.8 | 24.5 ± 1.8 | -9.3 pp, -28% |
| mmlu_redux(OOD) | 52.5 | 47.8 ± 1.9 | -4.7 pp, -9% |
| gpqa_diamond(OOD) | 7.6 | 31.6 ± 0.8 | +24.1 pp, +318% |
| InD Average | 83.8 | 66.8 ± 1.6 | -17.0 pp, -20% |
| OOD | 31.4 | 34.7 ± 0.4 | +3.3 pp, +10% |
| ALL | 68.1 | 57.2 ± 1.2 | -10.9 pp, -16% |
pp) and the relative percent change (tuned − base) / base × 100 (rel %, shown as n/a when base accuracy is 0).| Dataset | Base | Tuned (mean ± std) | Reduction % |
|---|---|---|---|
| gsm8k | 4450 | 1 ± 0 | -100% |
| arc_challenge | 3157 | 1 ± 0 | -100% |
| arc_easy | 1871 | 1 ± 0 | -100% |
| commonsenseqa | 3949 | 1 ± 0 | -100% |
| openbookqa | 3378 | 1 ± 0 | -100% |
| qasc | 3932 | 1 ± 0 | -100% |
| sciq | 1944 | 1 ± 0 | -100% |
| mmlu_pro(OOD) | 6582 | 1 ± 0 | -100% |
| mmlu_redux(OOD) | 5589 | 1 ± 0 | -100% |
| gpqa_diamond(OOD) | 8001 | 1 ± 0 | -100% |
| InD Average | 3240 | 1 ± 0 | -100% |
| OOD | 6720 | 1 ± 0 | -100% |
| ALL | 4281 | 1 ± 0 | -100% |
+, means the tuned model generates more tokens).