Compumacy/

toolcall_bench

💾 Github   |    📄 Paper

When2Call is a benchmark designed to evaluate tool-calling decision-making for large language models (LLMs), including when to generate a tool call, when to ask follow-up questions, when to admit the question can't be answered with the tools provided, and what to do if the question seems to require tool use but a tool call can't be made. We find that state-of-the-art tool-calling LMs show significant room for… See the full description on the dataset page: https://huggingface.co/datasets/Compumacy/toolcall_bench.