This is the source (benchmark dataset, test cases, and evaluation framework) of MegaBugFix, a large-scale benchmark to evaluate bugfixing capabilities introduced in the paper Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking.
Our goal was to create a Python dataset with incorrect source files, along with test cases to verify their attempted fixes. This way, developers can evaluate bugfixing approaches (such as LLMs) on these files by… See the full description on the dataset page:
https://huggingface.co/datasets/szalontaib/MegaBugFix-benchmark.
