BigCodeBench-Complete: Code Completion based on the structured docstrings. BigCodeBench-Instruct: Code Generation based on the NL-oriented instructions.

The overall statistics of the dataset are as follows:

Complete Instruct

Task

1140 1140

Avg. Test Cases

5.6 5.6

Avg. Coverage

99% 99%

Avg. Prompt Char.

1112.5 663.2

Avg. Prompt Line

33.5 11.7

Avg. Prompt Char. (Code)

1112.5 124.0

Avg. Solution Char.… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/bigcodebench.