SimpleQA Verified is a 1,000-prompt benchmark for reliably evaluating Large Language Models (LLMs) on short-form factuality and parametric knowledge. The authors from Google DeepMind and Google Research address various limitations of SimpleQA, originally designed by Wei et al. (2024) at OpenAI, including noisy and incorrect labels, topical biases, and question redundancy.
SimpleQA Verified was created to provide the research community with a more precise instrument to track genuine progress in… See the full description on the dataset page:
https://huggingface.co/datasets/stalkermustang/SimpleQA-Verified.
