| Benchmark | Mode | Sampled | Accuracy |
|---|---|---|---|
| MMLU | Sample | 1000/14042 | 79.5% |
| TRUTHFULQA | Full | 817/817 | 85.9% |
| GSM8K | Sample | 100/1319 | 94.0% |
| HUMANEVAL | Full | 164/164 | 95.1% |
| LIVECODEBENCH | Sample | 100/1055 | 42.0% |
| Benchmark | Accuracy | Correct | Total | Time(s) |
|---|---|---|---|---|
| MMLU | 79.5% | 795 | 1000 | 1271.4 |
| TRUTHFULQA | 85.9% | 702 | 817 | 501.8 |
| GSM8K | 94.0% | 94 | 100 | 261.1 |
| HUMANEVAL | 95.1% | 156 | 164 | 440.4 |
| LIVECODEBENCH | 42.0% | 42 | 100 | 5989.2 |
