Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 25 | acc | 0.1741 | ± | 0.0111 |
| none | 25 | acc_norm | 0.2304 | ± | 0.0123 | ||
| truthfulqa_mc2 | 2 | none | 0 | acc | 0.4616 | ± | 0.0156 |
| winogrande | 1 | none | 5 | acc | 0.5107 | ± | 0.014 |
| hellaswag | 1 | none | 10 | acc | 0.2753 | ± | 0.0045 |
| none | 10 | acc_norm | 0.2857 | ± | 0.0045 | ||
| gsm8k | 3 | strict-match | 5 | exact_match | 0.0061 | ± | 0.0021 |
| flexible-extract | 5 | exact_match | 0.0129 | ± | 0.0031 |
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| world_religions | 0 | none | 5 | acc | 0.2222 | ± | 0.0319 |
| virology | 0 | none | 5 | acc | 0.1988 | ± | 0.0311 |
| us_foreign_policy | 0 | none | 5 | acc | 0.2300 | ± | 0.0423 |
| sociology | 0 | none | 5 | acc | 0.2338 | ± | 0.0299 |
| security_studies | 0 | none | 5 | acc | 0.3673 | ± | 0.0309 |
| public_relations | 0 | none | 5 | acc | 0.2273 | ± | 0.0401 |
| professional_psychology | 0 | none | 5 | acc | 0.2402 | ± | 0.0173 |
| professional_medicine | 0 | none | 5 | acc | 0.4265 | ± | 0.0300 |
| professional_law | 0 | none | 5 | acc | 0.2419 | ± | 0.0109 |
| professional_accounting | 0 | none | 5 | acc | 0.2589 | ± | 0.0261 |
| prehistory | 0 | none | 5 | acc | 0.2716 | ± | 0.0247 |
| philosophy | 0 | none | 5 | acc | 0.2412 | ± | 0.0243 |
| nutrition | 0 | none | 5 | acc | 0.2516 | ± | 0.0248 |
| moral_scenarios | 0 | none | 5 | acc | 0.2514 | ± | 0.0145 |
| moral_disputes | 0 | none | 5 | acc | 0.2139 | ± | 0.0221 |
| miscellaneous | 0 | none | 5 | acc | 0.2644 | ± | 0.0158 |
| medical_genetics | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| marketing | 0 | none | 5 | acc | 0.1923 | ± | 0.0258 |
| management | 0 | none | 5 | acc | 0.1942 | ± | 0.0392 |
| machine_learning | 0 | none | 5 | acc | 0.2500 | ± | 0.0411 |
| logical_fallacies | 0 | none | 5 | acc | 0.2638 | ± | 0.0346 |
| jurisprudence | 0 | none | 5 | acc | 0.1759 | ± | 0.0368 |
| international_law | 0 | none | 5 | acc | 0.3554 | ± | 0.0437 |
| human_sexuality | 0 | none | 5 | acc | 0.2443 | ± | 0.0377 |
| human_aging | 0 | none | 5 | acc | 0.1928 | ± | 0.0265 |
| high_school_world_history | 0 | none | 5 | acc | 0.2700 | ± | 0.0289 |
| high_school_us_history | 0 | none | 5 | acc | 0.2990 | ± | 0.0321 |
| high_school_statistics | 0 | none | 5 | acc | 0.4074 | ± | 0.0335 |
| high_school_psychology | 0 | none | 5 | acc | 0.2422 | ± | 0.0184 |
| high_school_physics | 0 | none | 5 | acc | 0.2053 | ± | 0.0330 |
| high_school_microeconomics | 0 | none | 5 | acc | 0.2479 | ± | 0.0280 |
| high_school_mathematics | 0 | none | 5 | acc | 0.2815 | ± | 0.0274 |
| high_school_macroeconomics | 0 | none | 5 | acc | 0.2128 | ± | 0.0208 |
| high_school_government_and_politics | 0 | none | 5 | acc | 0.2435 | ± | 0.0310 |
| high_school_geography | 0 | none | 5 | acc | 0.3232 | ± | 0.0333 |
| high_school_european_history | 0 | none | 5 | acc | 0.2848 | ± | 0.0352 |
| high_school_computer_science | 0 | none | 5 | acc | 0.2800 | ± | 0.0451 |
| high_school_chemistry | 0 | none | 5 | acc | 0.2906 | ± | 0.0319 |
| high_school_biology | 0 | none | 5 | acc | 0.3032 | ± | 0.0261 |
| global_facts | 0 | none | 5 | acc | 0.1600 | ± | 0.0368 |
| formal_logic | 0 | none | 5 | acc | 0.1429 | ± | 0.0313 |
| elementary_mathematics | 0 | none | 5 | acc | 0.2434 | ± | 0.0221 |
| electrical_engineering | 0 | none | 5 | acc | 0.2483 | ± | 0.0360 |
| econometrics | 0 | none | 5 | acc | 0.2544 | ± | 0.0410 |
| conceptual_physics | 0 | none | 5 | acc | 0.3064 | ± | 0.0301 |
| computer_security | 0 | none | 5 | acc | 0.1700 | ± | 0.0378 |
| college_physics | 0 | none | 5 | acc | 0.2745 | ± | 0.0444 |
| college_medicine | 0 | none | 5 | acc | 0.2601 | ± | 0.0335 |
| college_mathematics | 0 | none | 5 | acc | 0.2500 | ± | 0.0435 |
| college_computer_science | 0 | none | 5 | acc | 0.2900 | ± | 0.0456 |
| college_chemistry | 0 | none | 5 | acc | 0.2400 | ± | 0.0429 |
| college_biology | 0 | none | 5 | acc | 0.2500 | ± | 0.0362 |
| clinical_knowledge | 0 | none | 5 | acc | 0.2075 | ± | 0.0250 |
| business_ethics | 0 | none | 5 | acc | 0.2000 | ± | 0.0402 |
| astronomy | 0 | none | 5 | acc | 0.1974 | ± | 0.0324 |
| anatomy | 0 | none | 5 | acc | 0.3185 | ± | 0.0402 |
| abstract_algebra | 0 | none | 5 | acc | 0.2300 | ± | 0.0423 |