Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| mmlu | N/A | none | 0 | acc | ↑ | 0.8152 | ± | 0.0031 |
| - abstract_algebra | 0 | none | 5 | acc | ↑ | 0.5600 | ± | 0.0499 |
| - anatomy | 0 | none | 5 | acc | ↑ | 0.8074 | ± | 0.0341 |
| - astronomy | 0 | none | 5 | acc | ↑ | 0.8947 | ± | 0.0250 |
| - business_ethics | 0 | none | 5 | acc | ↑ | 0.8300 | ± | 0.0378 |
| - clinical_knowledge | 0 | none | 5 | acc | ↑ | 0.8453 | ± | 0.0223 |
| - college_biology | 0 | none | 5 | acc | ↑ | 0.9375 | ± | 0.0202 |
| - college_chemistry | 0 | none | 5 | acc | ↑ | 0.5700 | ± | 0.0498 |
| - college_computer_science | 0 | none | 5 | acc | ↑ | 0.6700 | ± | 0.0473 |
| - college_mathematics | 0 | none | 5 | acc | ↑ | 0.4900 | ± | 0.0502 |
| - college_medicine | 0 | none | 5 | acc | ↑ | 0.7803 | ± | 0.0316 |
| - college_physics | 0 | none | 5 | acc | ↑ | 0.5980 | ± | 0.0488 |
| - computer_security | 0 | none | 5 | acc | ↑ | 0.8400 | ± | 0.0368 |
| - conceptual_physics | 0 | none | 5 | acc | ↑ | 0.8298 | ± | 0.0246 |
| - econometrics | 0 | none | 5 | acc | ↑ | 0.7281 | ± | 0.0419 |
| - electrical_engineering | 0 | none | 5 | acc | ↑ | 0.7931 | ± | 0.0338 |
| - elementary_mathematics | 0 | none | 5 | acc | ↑ | 0.7487 | ± | 0.0223 |
| - formal_logic | 0 | none | 5 | acc | ↑ | 0.6667 | ± | 0.0422 |
| - global_facts | 0 | none | 5 | acc | ↑ | 0.5900 | ± | 0.0494 |
| - high_school_biology | 0 | none | 5 | acc | ↑ | 0.9323 | ± | 0.0143 |
| - high_school_chemistry | 0 | none | 5 | acc | ↑ | 0.7488 | ± | 0.0305 |
| - high_school_computer_science | 0 | none | 5 | acc | ↑ | 0.9200 | ± | 0.0273 |
| - high_school_european_history | 0 | none | 5 | acc | ↑ | 0.8727 | ± | 0.0260 |
| - high_school_geography | 0 | none | 5 | acc | ↑ | 0.9343 | ± | 0.0176 |
| - high_school_government_and_politics | 0 | none | 5 | acc | ↑ | 0.9741 | ± | 0.0115 |
| - high_school_macroeconomics | 0 | none | 5 | acc | ↑ | 0.8667 | ± | 0.0172 |
| - high_school_mathematics | 0 | none | 5 | acc | ↑ | 0.5519 | ± | 0.0303 |
| - high_school_microeconomics | 0 | none | 5 | acc | ↑ | 0.9202 | ± | 0.0176 |
| - high_school_physics | 0 | none | 5 | acc | ↑ | 0.6623 | ± | 0.0386 |
| - high_school_psychology | 0 | none | 5 | acc | ↑ | 0.9486 | ± | 0.0095 |
| - high_school_statistics | 0 | none | 5 | acc | ↑ | 0.7407 | ± | 0.0299 |
| - high_school_us_history | 0 | none | 5 | acc | ↑ | 0.9461 | ± | 0.0159 |
| - high_school_world_history | 0 | none | 5 | acc | ↑ | 0.9283 | ± | 0.0168 |
| - human_aging | 0 | none | 5 | acc | ↑ | 0.8386 | ± | 0.0247 |
| - human_sexuality | 0 | none | 5 | acc | ↑ | 0.8931 | ± | 0.0271 |
| - humanities | N/A | none | 5 | acc | ↑ | 0.7921 | ± | 0.0057 |
| - international_law | 0 | none | 5 | acc | ↑ | 0.9256 | ± | 0.0240 |
| - jurisprudence | 0 | none | 5 | acc | ↑ | 0.8796 | ± | 0.0315 |
| - logical_fallacies | 0 | none | 5 | acc | ↑ | 0.8528 | ± | 0.0278 |
| - machine_learning | 0 | none | 5 | acc | ↑ | 0.6786 | ± | 0.0443 |
| - management | 0 | none | 5 | acc | ↑ | 0.9417 | ± | 0.0232 |
| - marketing | 0 | none | 5 | acc | ↑ | 0.9316 | ± | 0.0165 |
| - medical_genetics | 0 | none | 5 | acc | ↑ | 0.9100 | ± | 0.0288 |
| - miscellaneous | 0 | none | 5 | acc | ↑ | 0.9208 | ± | 0.0097 |
| - moral_disputes | 0 | none | 5 | acc | ↑ | 0.8439 | ± | 0.0195 |
| - moral_scenarios | 0 | none | 5 | acc | ↑ | 0.8089 | ± | 0.0131 |
| - nutrition | 0 | none | 5 | acc | ↑ | 0.9052 | ± | 0.0168 |
| - other | N/A | none | 5 | acc | ↑ | 0.8423 | ± | 0.0062 |
| - philosophy | 0 | none | 5 | acc | ↑ | 0.8360 | ± | 0.0210 |
| - prehistory | 0 | none | 5 | acc | ↑ | 0.8920 | ± | 0.0173 |
| - professional_accounting | 0 | none | 5 | acc | ↑ | 0.6631 | ± | 0.0282 |
| - professional_law | 0 | none | 5 | acc | ↑ | 0.6649 | ± | 0.0121 |
| - professional_medicine | 0 | none | 5 | acc | ↑ | 0.8971 | ± | 0.0185 |
| - professional_psychology | 0 | none | 5 | acc | ↑ | 0.8578 | ± | 0.0141 |
| - public_relations | 0 | none | 5 | acc | ↑ | 0.7455 | ± | 0.0417 |
| - security_studies | 0 | none | 5 | acc | ↑ | 0.8408 | ± | 0.0234 |
| - social_sciences | N/A | none | 5 | acc | ↑ | 0.8898 | ± | 0.0056 |
| - sociology | 0 | none | 5 | acc | ↑ | 0.9254 | ± | 0.0186 |
| - stem | N/A | none | 5 | acc | ↑ | 0.7501 | ± | 0.0074 |
| - us_foreign_policy | 0 | none | 5 | acc | ↑ | 0.9200 | ± | 0.0273 |
| - virology | 0 | none | 5 | acc | ↑ | 0.5663 | ± | 0.0386 |
| - world_religions | 0 | none | 5 | acc | ↑ | 0.9064 | ± | 0.0223 |
| Groups | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| mmlu | N/A | none | 0 | acc | ↑ | 0.8152 | ± | 0.0031 |
| - humanities | N/A | none | 5 | acc | ↑ | 0.7921 | ± | 0.0057 |
| - other | N/A | none | 5 | acc | ↑ | 0.8423 | ± | 0.0062 |
| - social_sciences | N/A | none | 5 | acc | ↑ | 0.8898 | ± | 0.0056 |
| - stem | N/A | none | 5 | acc | ↑ | 0.7501 | ± | 0.0074 |