FORTIS is a benchmark for evaluating AI agent safety in skill and tool selection. It measures whether LLM agents select minimally-privileged capabilities when multiple valid options exist.
Modern LLM agents operate through a skill layer that mediates between user intent and task execution. FORTIS evaluates two critical safety questions:
Task 1: Skill Selection - Does the agent select the minimally sufficient skill… See the full description on the dataset page:
https://huggingface.co/datasets/ShawnLi02/FORTIS_Agent_Skill_Safety.
