Atypical answer prediction — predict from activations whether the model will give the majority or minority answer. Based on ICLR 2026 'When Just Read the Chain of Thought Fails' Section 2.5. 200 rollouts per question with temperature=0.6, filter to 10-35% minority rate, 15 majority + 15 minority per question CoT-length-matched. Moral dilemmas (Daily Dilemmas + Scruples) to avoid correctness confounder, plus PIQA OOD test. 50/50 balanced… See the full description on the dataset page:
https://huggingface.co/datasets/ceselder/cot-oracle-eval-atypical-answer.
