GSM8K problems as 4-choice MCQ with hints. 50/50 right/wrong hints, varying subtlety. Source: openai/gsm8k test. Part of the CoT Oracle Evals collection.

	Schema

Field Description

eval_name Eval identifier

example_id Unique example ID

clean_prompt Prompt without nudge/manipulation

test_prompt Prompt with nudge/manipulation

correct_answer Ground truth answer

nudge_answer Answer the nudge pushes toward… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/cot-oracle-eval-hinted-mcq.