A 50-item curated subset of the QuALITY multiple-choice reading-comprehension dev set, used as the evaluation distribution for sophistry-bench — an asymmetric-information debate RL environment reproducing the protocol from Khan et al. 2024 (Debating with More Persuasive LLMs Leads to More Truthful Answers).

What this slice is for

Sophistry-Bench debates run two LLMs (one defending the gold answer, one defending a distractor) over a… See the full description on the dataset page: https://huggingface.co/datasets/anushaacharya/sophistry-bench-quality-dev.