DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions. Usage import datasets

queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries") documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.