Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 25 | acc | 0.1800 | ± | 0.0112 |
| none | 25 | acc_norm | 0.2218 | ± | 0.0121 | ||
| truthfulqa_mc2 | 2 | none | 0 | acc | 0.4504 | ± | 0.0155 |
| winogrande | 1 | none | 5 | acc | 0.502 | ± | 0.0141 |
| hellaswag | 1 | none | 10 | acc | 0.2726 | ± | 0.0044 |
| none | 10 | acc_norm | 0.2797 | ± | 0.0045 | ||
| gsm8k | 3 | strict-match | 5 | exact_match | 0.0091 | ± | 0.0026 |
| flexible-extract | 5 | exact_match | 0.0152 | ± | 0.0034 |
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| world_religions | 0 | none | 5 | acc | 0.2164 | ± | 0.0316 |
| virology | 0 | none | 5 | acc | 0.1988 | ± | 0.0311 |
| us_foreign_policy | 0 | none | 5 | acc | 0.3200 | ± | 0.0469 |
| sociology | 0 | none | 5 | acc | 0.2338 | ± | 0.0299 |
| security_studies | 0 | none | 5 | acc | 0.2367 | ± | 0.0272 |
| public_relations | 0 | none | 5 | acc | 0.2455 | ± | 0.0412 |
| professional_psychology | 0 | none | 5 | acc | 0.2467 | ± | 0.0174 |
| professional_medicine | 0 | none | 5 | acc | 0.4412 | ± | 0.0302 |
| professional_law | 0 | none | 5 | acc | 0.2666 | ± | 0.0113 |
| professional_accounting | 0 | none | 5 | acc | 0.2553 | ± | 0.0260 |
| prehistory | 0 | none | 5 | acc | 0.2870 | ± | 0.0252 |
| philosophy | 0 | none | 5 | acc | 0.3023 | ± | 0.0261 |
| nutrition | 0 | none | 5 | acc | 0.2157 | ± | 0.0236 |
| moral_scenarios | 0 | none | 5 | acc | 0.2469 | ± | 0.0144 |
| moral_disputes | 0 | none | 5 | acc | 0.2168 | ± | 0.0222 |
| miscellaneous | 0 | none | 5 | acc | 0.2720 | ± | 0.0159 |
| medical_genetics | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| marketing | 0 | none | 5 | acc | 0.1966 | ± | 0.0260 |
| management | 0 | none | 5 | acc | 0.1942 | ± | 0.0392 |
| machine_learning | 0 | none | 5 | acc | 0.2411 | ± | 0.0406 |
| logical_fallacies | 0 | none | 5 | acc | 0.2331 | ± | 0.0332 |
| jurisprudence | 0 | none | 5 | acc | 0.2130 | ± | 0.0396 |
| international_law | 0 | none | 5 | acc | 0.3554 | ± | 0.0437 |
| human_sexuality | 0 | none | 5 | acc | 0.2824 | ± | 0.0395 |
| human_aging | 0 | none | 5 | acc | 0.3139 | ± | 0.0311 |
| high_school_world_history | 0 | none | 5 | acc | 0.2489 | ± | 0.0281 |
| high_school_us_history | 0 | none | 5 | acc | 0.2157 | ± | 0.0289 |
| high_school_statistics | 0 | none | 5 | acc | 0.4722 | ± | 0.0340 |
| high_school_psychology | 0 | none | 5 | acc | 0.2055 | ± | 0.0173 |
| high_school_physics | 0 | none | 5 | acc | 0.3113 | ± | 0.0378 |
| high_school_microeconomics | 0 | none | 5 | acc | 0.2857 | ± | 0.0293 |
| high_school_mathematics | 0 | none | 5 | acc | 0.2667 | ± | 0.0270 |
| high_school_macroeconomics | 0 | none | 5 | acc | 0.2179 | ± | 0.0209 |
| high_school_government_and_politics | 0 | none | 5 | acc | 0.2539 | ± | 0.0314 |
| high_school_geography | 0 | none | 5 | acc | 0.2475 | ± | 0.0307 |
| high_school_european_history | 0 | none | 5 | acc | 0.2606 | ± | 0.0343 |
| high_school_computer_science | 0 | none | 5 | acc | 0.2000 | ± | 0.0402 |
| high_school_chemistry | 0 | none | 5 | acc | 0.2857 | ± | 0.0318 |
| high_school_biology | 0 | none | 5 | acc | 0.3161 | ± | 0.0265 |
| global_facts | 0 | none | 5 | acc | 0.3100 | ± | 0.0465 |
| formal_logic | 0 | none | 5 | acc | 0.1508 | ± | 0.0320 |
| elementary_mathematics | 0 | none | 5 | acc | 0.2566 | ± | 0.0225 |
| electrical_engineering | 0 | none | 5 | acc | 0.2414 | ± | 0.0357 |
| econometrics | 0 | none | 5 | acc | 0.2456 | ± | 0.0405 |
| conceptual_physics | 0 | none | 5 | acc | 0.2979 | ± | 0.0299 |
| computer_security | 0 | none | 5 | acc | 0.1600 | ± | 0.0368 |
| college_physics | 0 | none | 5 | acc | 0.3039 | ± | 0.0458 |
| college_medicine | 0 | none | 5 | acc | 0.2254 | ± | 0.0319 |
| college_mathematics | 0 | none | 5 | acc | 0.2300 | ± | 0.0423 |
| college_computer_science | 0 | none | 5 | acc | 0.3300 | ± | 0.0473 |
| college_chemistry | 0 | none | 5 | acc | 0.4200 | ± | 0.0496 |
| college_biology | 0 | none | 5 | acc | 0.2500 | ± | 0.0362 |
| clinical_knowledge | 0 | none | 5 | acc | 0.2189 | ± | 0.0254 |
| business_ethics | 0 | none | 5 | acc | 0.2100 | ± | 0.0409 |
| astronomy | 0 | none | 5 | acc | 0.1711 | ± | 0.0306 |
| anatomy | 0 | none | 5 | acc | 0.3333 | ± | 0.0407 |
| abstract_algebra | 0 | none | 5 | acc | 0.2700 | ± | 0.0446 |