Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 25 | acc | 0.1792 | ± | 0.0112 |
| none | 25 | acc_norm | 0.2065 | ± | 0.0118 | ||
| truthfulqa_mc2 | 2 | none | 0 | acc | 0.4553 | ± | 0.0154 |
| winogrande | 1 | none | 5 | acc | 0.4972 | ± | 0.0141 |
| hellaswag | 1 | none | 10 | acc | 0.2703 | ± | 0.0044 |
| none | 10 | acc_norm | 0.2796 | ± | 0.0045 | ||
| gsm8k | 3 | strict-match | 5 | exact_match | 0.0000 | ± | 0.0000 |
| flexible-extract | 5 | exact_match | 0.0144 | ± | 0.0033 |
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| world_religions | 0 | none | 5 | acc | 0.2573 | ± | 0.0335 |
| virology | 0 | none | 5 | acc | 0.2831 | ± | 0.0351 |
| us_foreign_policy | 0 | none | 5 | acc | 0.2500 | ± | 0.0435 |
| sociology | 0 | none | 5 | acc | 0.2438 | ± | 0.0304 |
| security_studies | 0 | none | 5 | acc | 0.2327 | ± | 0.0270 |
| public_relations | 0 | none | 5 | acc | 0.2273 | ± | 0.0401 |
| professional_psychology | 0 | none | 5 | acc | 0.2500 | ± | 0.0175 |
| professional_medicine | 0 | none | 5 | acc | 0.4485 | ± | 0.0302 |
| professional_law | 0 | none | 5 | acc | 0.2458 | ± | 0.0110 |
| professional_accounting | 0 | none | 5 | acc | 0.2624 | ± | 0.0262 |
| prehistory | 0 | none | 5 | acc | 0.2130 | ± | 0.0228 |
| philosophy | 0 | none | 5 | acc | 0.1929 | ± | 0.0224 |
| nutrition | 0 | none | 5 | acc | 0.2222 | ± | 0.0238 |
| moral_scenarios | 0 | none | 5 | acc | 0.2380 | ± | 0.0142 |
| moral_disputes | 0 | none | 5 | acc | 0.2486 | ± | 0.0233 |
| miscellaneous | 0 | none | 5 | acc | 0.2644 | ± | 0.0158 |
| medical_genetics | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| marketing | 0 | none | 5 | acc | 0.1752 | ± | 0.0249 |
| management | 0 | none | 5 | acc | 0.1748 | ± | 0.0376 |
| machine_learning | 0 | none | 5 | acc | 0.2500 | ± | 0.0411 |
| logical_fallacies | 0 | none | 5 | acc | 0.2945 | ± | 0.0358 |
| jurisprudence | 0 | none | 5 | acc | 0.2593 | ± | 0.0424 |
| international_law | 0 | none | 5 | acc | 0.2479 | ± | 0.0394 |
| human_sexuality | 0 | none | 5 | acc | 0.2595 | ± | 0.0384 |
| human_aging | 0 | none | 5 | acc | 0.2466 | ± | 0.0289 |
| high_school_world_history | 0 | none | 5 | acc | 0.2911 | ± | 0.0296 |
| high_school_us_history | 0 | none | 5 | acc | 0.2794 | ± | 0.0315 |
| high_school_statistics | 0 | none | 5 | acc | 0.4722 | ± | 0.0340 |
| high_school_psychology | 0 | none | 5 | acc | 0.1927 | ± | 0.0169 |
| high_school_physics | 0 | none | 5 | acc | 0.1987 | ± | 0.0326 |
| high_school_microeconomics | 0 | none | 5 | acc | 0.2227 | ± | 0.0270 |
| high_school_mathematics | 0 | none | 5 | acc | 0.2667 | ± | 0.0270 |
| high_school_macroeconomics | 0 | none | 5 | acc | 0.2103 | ± | 0.0207 |
| high_school_government_and_politics | 0 | none | 5 | acc | 0.2435 | ± | 0.0310 |
| high_school_geography | 0 | none | 5 | acc | 0.1717 | ± | 0.0269 |
| high_school_european_history | 0 | none | 5 | acc | 0.2485 | ± | 0.0337 |
| high_school_computer_science | 0 | none | 5 | acc | 0.2700 | ± | 0.0446 |
| high_school_chemistry | 0 | none | 5 | acc | 0.2906 | ± | 0.0319 |
| high_school_biology | 0 | none | 5 | acc | 0.2774 | ± | 0.0255 |
| global_facts | 0 | none | 5 | acc | 0.1600 | ± | 0.0368 |
| formal_logic | 0 | none | 5 | acc | 0.1508 | ± | 0.0320 |
| elementary_mathematics | 0 | none | 5 | acc | 0.2540 | ± | 0.0224 |
| electrical_engineering | 0 | none | 5 | acc | 0.2414 | ± | 0.0357 |
| econometrics | 0 | none | 5 | acc | 0.2544 | ± | 0.0410 |
| conceptual_physics | 0 | none | 5 | acc | 0.2638 | ± | 0.0288 |
| computer_security | 0 | none | 5 | acc | 0.2600 | ± | 0.0441 |
| college_physics | 0 | none | 5 | acc | 0.2157 | ± | 0.0409 |
| college_medicine | 0 | none | 5 | acc | 0.2081 | ± | 0.0310 |
| college_mathematics | 0 | none | 5 | acc | 0.2300 | ± | 0.0423 |
| college_computer_science | 0 | none | 5 | acc | 0.3100 | ± | 0.0465 |
| college_chemistry | 0 | none | 5 | acc | 0.2000 | ± | 0.0402 |
| college_biology | 0 | none | 5 | acc | 0.2431 | ± | 0.0359 |
| clinical_knowledge | 0 | none | 5 | acc | 0.2415 | ± | 0.0263 |
| business_ethics | 0 | none | 5 | acc | 0.1600 | ± | 0.0368 |
| astronomy | 0 | none | 5 | acc | 0.1776 | ± | 0.0311 |
| anatomy | 0 | none | 5 | acc | 0.3407 | ± | 0.0409 |
| abstract_algebra | 0 | none | 5 | acc | 0.2200 | ± | 0.0416 |