Can a language model change its answer when the rules of modal logic change? Each example contains the same premises and conclusion under two semantic specifications. Only one rule about possible worlds or objects changes, and the correct answer changes with it. Automated theorem provers verify every label. This dataset accompanies Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?

	Dataset subsets… See the full description on the dataset page: https://huggingface.co/datasets/sileod/modal-semantics-reasoning.