exception: data did not match any variant of untagged enum modelwrapper at line 1251003 column 3pip install --upgrade "transformers>=4.45"| Metric | Value |
|---|---|
| Avg. | 16.60 |
| IFEval (0-Shot) | 54.74 |
| BBH (3-Shot) | 19.52 |
| MATH Lvl 5 (4-Shot) | 5.29 |
| GPQA (0-shot) | 0.11 |
| MuSR (0-shot) | 3.96 |
| MMLU-PRO (5-shot) | 15.96 |