BIG-Bench Hard with CoT few-shot exemplars introduced in the repository:
https://github.com/suzgunmirac/BIG-Bench-Hard.
BBH_INSTRUCTIONS = {
"boolean_expressions": "Evaluate the truth value of a random Boolean expression consisting of Boolean constants (
True,
False) and basic Boolean operators (
and,
or,
not).",
"causal_judgement": "Given a short story (involving moral, intentional, or counterfactual analysis)… See the full description on the dataset page:
https://huggingface.co/datasets/affjljoo3581/bbh_cot.