Benchmark for evaluating how effectively PDF parsing solutions extract mathematical formulas and tables from documents.
We generate synthetic PDFs with diverse formatting scenarios, parse them with different parsers, and score the extracted content using LLM-as-a-Judge. This semantic evaluation approach substantially outperforms traditional metrics in agreement with human judgment.
