BatchSafeBench is a comprehensive benchmark introduced in the ACL 2025 Findings paper (
https://arxiv.org/abs/2503.15551), designed to systematically study batch prompting vulnerabilities.
Batch prompting, which combines a batch of multiple queries sharing the same context in one inference, has emerged as a promising solution to reduce inference costs. However, our study reveals a significant security vulnerability in batch prompting: malicious users can inject attack… See the full description on the dataset page:
https://huggingface.co/datasets/MurongYue/BatchSafeBench.
