We use Stdio input/output format here. For example, for the task to calculate the sum of a list, the input and output are in the following format:
input = "5\n1 2 3 4 5\n"
output = "15"
CodeContests and CodeForces are using this format, however, MBPP and part of LiveCodeBench are using functional input/output format, such like
assert sum_function([1, 2, 3, 4, 5]) == 15
In this project, we have converted the the functional format to the Stdio format to achieve consistency.
Paper | Code… See the full description on the dataset page:
https://huggingface.co/datasets/Gen-Verse/LiveBench-ReasonFlux.
