The dataset contains 5 splits. The clean split is a merged version of 6 manually annotated datasets into MMLU format. The original datasets are:
OpenBookQA (general)
ARC-Challenge (general)
ARC-Easy (general)
TruthfulQA (mix)
MedQA (medical)
MathQA (math)
Each split contains a corruption applied to the initial correct multiple choice question. Current corruptions are:
Strategy: randomly select a wrong… See the full description on the dataset page:
https://huggingface.co/datasets/edinburgh-dawg/labelchaos.
