To make the benchmark more lightweight and minimize resource consumption, we have introduced the Multi-SWE-bench mini.
This version contains 400 instances, with 50 instances per language (Python, Java, TypeScript, JavaScript, Go, Rust, C, and C++). It covers a range of difficulty levels—easy, medium, and hard—ensuring a balanced and efficient evaluation across multiple languages without excessive resource usage.The leaderboard can be found at:… See the full description on the dataset page:
https://huggingface.co/datasets/ByteDance-Seed/Multi-SWE-bench_mini.
