This repo contains the data from our paper "QAPyramid: Fine-grained Evaluation of Content Selection for Text Summarization".
Please visit here for more details of this project.
QAPyramid is built on top of 500 examples from the test set of the CNNDM English news summarization dataset.
Humans (crowdsourced workers) decomposed each reference summary into QA pairs following the QA-SRL framework.
On a 50-example subset, we get model-generated summaries from 10 summarization… See the full description on the dataset page:
https://huggingface.co/datasets/shiyue/QAPyramid.
