Views
No views yet
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 25 | acc | 0.1775 | ± | 0.0112 |
| none | 25 | acc_norm | 0.2065 | ± | 0.0118 | ||
| truthfulqa_mc2 | 2 | none | 0 | acc | 0.4633 | ± | 0.0155 |
| winogrande | 1 | none | 5 | acc | 0.5075 | ± | 0.0141 |
| hellaswag | 1 | none | 10 | acc | 0.2685 | ± | 0.0044 |
| none | 10 | acc_norm | 0.2746 | ± | 0.0045 | ||
| gsm8k | 3 | strict-match | 5 | exact_match | 0.0023 | ± | 0.0013 |
| flexible-extract | 5 | exact_match | 0.0152 | ± | 0.0034 |
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| world_religions | 0 | none | 5 | acc | 0.2047 | ± | 0.0309 |
| virology | 0 | none | 5 | acc | 0.1807 | ± | 0.0300 |
| us_foreign_policy | 0 | none | 5 | acc | 0.2700 | ± | 0.0446 |
| sociology | 0 | none | 5 | acc | 0.2488 | ± | 0.0306 |
| security_studies | 0 | none | 5 | acc | 0.3347 | ± | 0.0302 |
| public_relations | 0 | none | 5 | acc | 0.2273 | ± | 0.0401 |
| professional_psychology | 0 | none | 5 | acc | 0.2042 | ± | 0.0163 |
| professional_medicine | 0 | none | 5 | acc | 0.4485 | ± | 0.0302 |
| professional_law | 0 | none | 5 | acc | 0.2458 | ± | 0.0110 |
| professional_accounting | 0 | none | 5 | acc | 0.2163 | ± | 0.0246 |
| prehistory | 0 | none | 5 | acc | 0.2222 | ± | 0.0231 |
| philosophy | 0 | none | 5 | acc | 0.2379 | ± | 0.0242 |
| nutrition | 0 | none | 5 | acc | 0.2810 | ± | 0.0257 |
| moral_scenarios | 0 | none | 5 | acc | 0.2659 | ± | 0.0148 |
| moral_disputes | 0 | none | 5 | acc | 0.2428 | ± | 0.0231 |
| miscellaneous | 0 | none | 5 | acc | 0.2375 | ± | 0.0152 |
| medical_genetics | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| marketing | 0 | none | 5 | acc | 0.1966 | ± | 0.0260 |
| management | 0 | none | 5 | acc | 0.1553 | ± | 0.0359 |
| machine_learning | 0 | none | 5 | acc | 0.3304 | ± | 0.0446 |
| logical_fallacies | 0 | none | 5 | acc | 0.2331 | ± | 0.0332 |
| jurisprudence | 0 | none | 5 | acc | 0.2407 | ± | 0.0413 |
| international_law | 0 | none | 5 | acc | 0.3306 | ± | 0.0429 |
| human_sexuality | 0 | none | 5 | acc | 0.2595 | ± | 0.0384 |
| human_aging | 0 | none | 5 | acc | 0.2063 | ± | 0.0272 |
| high_school_world_history | 0 | none | 5 | acc | 0.2658 | ± | 0.0288 |
| high_school_us_history | 0 | none | 5 | acc | 0.2745 | ± | 0.0313 |
| high_school_statistics | 0 | none | 5 | acc | 0.4722 | ± | 0.0340 |
| high_school_psychology | 0 | none | 5 | acc | 0.2330 | ± | 0.0181 |
| high_school_physics | 0 | none | 5 | acc | 0.3311 | ± | 0.0384 |
| high_school_microeconomics | 0 | none | 5 | acc | 0.3403 | ± | 0.0308 |
| high_school_mathematics | 0 | none | 5 | acc | 0.2630 | ± | 0.0268 |
| high_school_macroeconomics | 0 | none | 5 | acc | 0.3205 | ± | 0.0237 |
| high_school_government_and_politics | 0 | none | 5 | acc | 0.3679 | ± | 0.0348 |
| high_school_geography | 0 | none | 5 | acc | 0.3283 | ± | 0.0335 |
| high_school_european_history | 0 | none | 5 | acc | 0.2606 | ± | 0.0343 |
| high_school_computer_science | 0 | none | 5 | acc | 0.2800 | ± | 0.0451 |
| high_school_chemistry | 0 | none | 5 | acc | 0.2956 | ± | 0.0321 |
| high_school_biology | 0 | none | 5 | acc | 0.3194 | ± | 0.0265 |
| global_facts | 0 | none | 5 | acc | 0.1600 | ± | 0.0368 |
| formal_logic | 0 | none | 5 | acc | 0.1825 | ± | 0.0346 |
| elementary_mathematics | 0 | none | 5 | acc | 0.2487 | ± | 0.0223 |
| electrical_engineering | 0 | none | 5 | acc | 0.2966 | ± | 0.0381 |
| econometrics | 0 | none | 5 | acc | 0.2632 | ± | 0.0414 |
| conceptual_physics | 0 | none | 5 | acc | 0.2553 | ± | 0.0285 |
| computer_security | 0 | none | 5 | acc | 0.1800 | ± | 0.0386 |
| college_physics | 0 | none | 5 | acc | 0.2451 | ± | 0.0428 |
| college_medicine | 0 | none | 5 | acc | 0.2312 | ± | 0.0321 |
| college_mathematics | 0 | none | 5 | acc | 0.3200 | ± | 0.0469 |
| college_computer_science | 0 | none | 5 | acc | 0.3000 | ± | 0.0461 |
| college_chemistry | 0 | none | 5 | acc | 0.1800 | ± | 0.0386 |
| college_biology | 0 | none | 5 | acc | 0.2778 | ± | 0.0375 |
| clinical_knowledge | 0 | none | 5 | acc | 0.2340 | ± | 0.0261 |
| business_ethics | 0 | none | 5 | acc | 0.2100 | ± | 0.0409 |
| astronomy | 0 | none | 5 | acc | 0.1776 | ± | 0.0311 |
| anatomy | 0 | none | 5 | acc | 0.2296 | ± | 0.0363 |
| abstract_algebra | 0 | none | 5 | acc | 0.2200 | ± | 0.0416 |