StateShiftBench is a synthetic benchmark for evaluating dynamic state invalidation in tool-using language model agents. It targets high-timeliness workflows where an agent observes state, reasons or plans, and then commits an irreversible action after the environment may have changed.
The benchmark contains 100 tasks across 10 production-relevant domains:
spot trading
database business operations
CI/CD release control
IAM and cloud security
Kubernetes resource… See the full description on the dataset page:
https://huggingface.co/datasets/xsx001/stateshiftbench-review.
