You can deploy this model and start earning money today!

whiss666/
SWT-bench_Lite_bm25_27k_zsb

Views

No views yet

Dataset Summary

SWT-bench Lite is subset of SWT-bench, a dataset that tests systems’ ability to reproduce GitHub issues automatically. The dataset collects 276 test Issue-Pull Request pairs from 11 popular Python GitHub projects. Evaluation is performed by unit test verification using pre- and post-PR behavior of the test suite with and without the model proposed tests.

📊🏆 Leaderboard

A public leaderboard for performance on SWT-bench is hosted at swtbench.com The… See the full description on the dataset page: https://huggingface.co/datasets/whiss666/SWT-bench_Lite_bm25_27k_zsb.