APPS is a benchmark for Python code generation, it includes 10,000 problems, which range from having simple oneline solutions to being substantial algorithmic challenges, for more details please refer to this paper:
https://arxiv.org/pdf/2105.09938.pdf.
