MapEval-API is created using MapQaTor.

from datasets import load_dataset

ds = load_dataset("MapEval/MapEval-API", name="benchmark")

for item in ds["test"]: # Start with a clear task description prompt = ( "You are a highly intelligent assistant. " "Answer the multiple-choice question by selecting the correct option.\n\n" "Question:\n" + item["question"] + "\n\n"… See the full description on the dataset page: https://huggingface.co/datasets/MapEval/MapEval-API.