Existing evaluation benchmarks of language models of code (code LMs) focus almost exclusively on whether the LMs can generate functionally-correct code. In real-world software engineering, developers think beyond functional correctness. They have requirements on "how" a functionality should be implemented to meet overall system design objectives like… See the full description on the dataset page:
https://huggingface.co/datasets/ManavSinghal157/NoFunEval.
