English wikipedia cleaned and deduplicated.
This dataset has also been decontaminated with respect to the following benchmarks based on n-gram overlap:
GLUE (dev set of SST-2, CoLA, QQP, WNLI, RTE, QNLI, MNLI; test set of MPRC)
SIQA, PIQA, QASC, CSQA, HellaSWAG (all dev set)
CONLL 2003
BLIMP
MAIN
BoolQ (dev set)
WinoGrande (dev set)
ANLI (test set)
ARC easy and challenge (test set)
RACE middle and high (test set)
MMLU (dev, val, and test sets)
MATH, GSM8K (test set)
HumanEval (test set)
GPQA… See the full description on the dataset page:
https://huggingface.co/datasets/Geralt-Targaryen/wikipedia.
