Can a reviewer find the defective lines in a change before it is merged? CoReDD Bench turns that question into a measurement by reconstructing labels from later corrections, auditing their error, and propagating the uncertainty into the metrics.
CoReDD Bench ships the labels, the uncertainty model, and the evaluation code. The repository histories and public GitHub review context these were reconstructed from are preserved separately in CoReDD Corpus.… See the full description on the dataset page:
https://huggingface.co/datasets/witrin/coredd-bench.
