Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 25 | acc | 0.1775 | ± | 0.0112 |
| none | 25 | acc_norm | 0.2150 | ± | 0.0120 | ||
| truthfulqa_mc2 | 2 | none | 0 | acc | 0.4571 | ± | 0.0154 |
| winogrande | 1 | none | 5 | acc | 0.5107 | ± | 0.014 |
| hellaswag | 1 | none | 10 | acc | 0.2735 | ± | 0.0044 |
| none | 10 | acc_norm | 0.2801 | ± | 0.0045 | ||
| gsm8k | 3 | strict-match | 5 | exact_match | 0.0030 | ± | 0.0015 |
| flexible-extract | 5 | exact_match | 0.0099 | ± | 0.0027 |
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| world_religions | 0 | none | 5 | acc | 0.2281 | ± | 0.0322 |
| virology | 0 | none | 5 | acc | 0.2289 | ± | 0.0327 |
| us_foreign_policy | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| sociology | 0 | none | 5 | acc | 0.2438 | ± | 0.0304 |
| security_studies | 0 | none | 5 | acc | 0.2327 | ± | 0.0270 |
| public_relations | 0 | none | 5 | acc | 0.2091 | ± | 0.0390 |
| professional_psychology | 0 | none | 5 | acc | 0.2516 | ± | 0.0176 |
| professional_medicine | 0 | none | 5 | acc | 0.4522 | ± | 0.0302 |
| professional_law | 0 | none | 5 | acc | 0.2484 | ± | 0.0110 |
| professional_accounting | 0 | none | 5 | acc | 0.2518 | ± | 0.0259 |
| prehistory | 0 | none | 5 | acc | 0.2654 | ± | 0.0246 |
| philosophy | 0 | none | 5 | acc | 0.2315 | ± | 0.0240 |
| nutrition | 0 | none | 5 | acc | 0.2059 | ± | 0.0232 |
| moral_scenarios | 0 | none | 5 | acc | 0.2380 | ± | 0.0142 |
| moral_disputes | 0 | none | 5 | acc | 0.2486 | ± | 0.0233 |
| miscellaneous | 0 | none | 5 | acc | 0.2874 | ± | 0.0162 |
| medical_genetics | 0 | none | 5 | acc | 0.2900 | ± | 0.0456 |
| marketing | 0 | none | 5 | acc | 0.2009 | ± | 0.0262 |
| management | 0 | none | 5 | acc | 0.1845 | ± | 0.0384 |
| machine_learning | 0 | none | 5 | acc | 0.2857 | ± | 0.0429 |
| logical_fallacies | 0 | none | 5 | acc | 0.3190 | ± | 0.0366 |
| jurisprudence | 0 | none | 5 | acc | 0.2685 | ± | 0.0428 |
| international_law | 0 | none | 5 | acc | 0.2149 | ± | 0.0375 |
| human_sexuality | 0 | none | 5 | acc | 0.2137 | ± | 0.0360 |
| human_aging | 0 | none | 5 | acc | 0.2466 | ± | 0.0289 |
| high_school_world_history | 0 | none | 5 | acc | 0.2616 | ± | 0.0286 |
| high_school_us_history | 0 | none | 5 | acc | 0.2402 | ± | 0.0300 |
| high_school_statistics | 0 | none | 5 | acc | 0.4722 | ± | 0.0340 |
| high_school_psychology | 0 | none | 5 | acc | 0.2128 | ± | 0.0175 |
| high_school_physics | 0 | none | 5 | acc | 0.2781 | ± | 0.0366 |
| high_school_microeconomics | 0 | none | 5 | acc | 0.3067 | ± | 0.0300 |
| high_school_mathematics | 0 | none | 5 | acc | 0.2630 | ± | 0.0268 |
| high_school_macroeconomics | 0 | none | 5 | acc | 0.2590 | ± | 0.0222 |
| high_school_government_and_politics | 0 | none | 5 | acc | 0.3005 | ± | 0.0331 |
| high_school_geography | 0 | none | 5 | acc | 0.3030 | ± | 0.0327 |
| high_school_european_history | 0 | none | 5 | acc | 0.2667 | ± | 0.0345 |
| high_school_computer_science | 0 | none | 5 | acc | 0.2900 | ± | 0.0456 |
| high_school_chemistry | 0 | none | 5 | acc | 0.2956 | ± | 0.0321 |
| high_school_biology | 0 | none | 5 | acc | 0.2871 | ± | 0.0257 |
| global_facts | 0 | none | 5 | acc | 0.2600 | ± | 0.0441 |
| formal_logic | 0 | none | 5 | acc | 0.1667 | ± | 0.0333 |
| elementary_mathematics | 0 | none | 5 | acc | 0.2566 | ± | 0.0225 |
| electrical_engineering | 0 | none | 5 | acc | 0.2414 | ± | 0.0357 |
| econometrics | 0 | none | 5 | acc | 0.2719 | ± | 0.0419 |
| conceptual_physics | 0 | none | 5 | acc | 0.3319 | ± | 0.0308 |
| computer_security | 0 | none | 5 | acc | 0.2000 | ± | 0.0402 |
| college_physics | 0 | none | 5 | acc | 0.1863 | ± | 0.0387 |
| college_medicine | 0 | none | 5 | acc | 0.1965 | ± | 0.0303 |
| college_mathematics | 0 | none | 5 | acc | 0.2700 | ± | 0.0446 |
| college_computer_science | 0 | none | 5 | acc | 0.2300 | ± | 0.0423 |
| college_chemistry | 0 | none | 5 | acc | 0.2000 | ± | 0.0402 |
| college_biology | 0 | none | 5 | acc | 0.2431 | ± | 0.0359 |
| clinical_knowledge | 0 | none | 5 | acc | 0.2226 | ± | 0.0256 |
| business_ethics | 0 | none | 5 | acc | 0.2100 | ± | 0.0409 |
| astronomy | 0 | none | 5 | acc | 0.1842 | ± | 0.0315 |
| anatomy | 0 | none | 5 | acc | 0.3407 | ± | 0.0409 |
| abstract_algebra | 0 | none | 5 | acc | 0.2400 | ± | 0.0429 |