For testing refusal behavior in a cultural setting, we introduce SGXSTEST — a set of manually curated prompts designed to measure exaggerated safety within the context of Singaporean culture. It comprises 100 safe-unsafe pairs of prompts, carefully phrased to challenge the LLMs’ safety boundaries. The dataset covers 10 categories of hazards (adapted from Röttger et al. (2023)), with 10 safe-unsafe prompt pairs in each category. These categories include homonyms, figurative… See the full description on the dataset page:
https://huggingface.co/datasets/walledai/SGXSTest.
