This dataset FM-Alpaca is the training set for the paper From Informal to Formal -- Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs. You may find the testing data (benchmark) here.
FM-Bench includes:
Six formal-verification-related tasks (See Figure 1 below):
Requirement Analysis (abbrev. ReqAna): given requirements and description of the verification or modeling goals, decomposing the goal into detailed verification… See the full description on the dataset page:
https://huggingface.co/datasets/fm-universe/FM-alpaca.
