Views
No views yet
| Bench | Fmt | Method | Recall@5 | Precision@5 |
|---|---|---|---|---|
| ProofNet | F | BM25 | 0.16% | 0.11% |
| F | DR | 35.52% | 22.89% | |
| F+IF | BM25 | 0.00% | 0.00% | |
| F+IF | DR | 32.47% | 20.32% | |
| Con-NF | F | BM25 | 4.41% | 2.37% |
| F | DR | 24.32% | 14.05% | |
| F+IF | BM25 | 9.86% | 6.95% | |
| F+IF | DR | 27.91% | 17.57% |
purewhite42/bm25_f: BM25 dependency retriever whose inputs are formatted using only formal declarations, based on Rank-BM25purewhite42/bm25_f_if: BM25 dependency retriever whose inputs are formatted using both formal declarations and informal descriptions, based on Rank-BM251python -m retriever.retrieve_bm25 \
2 --model_path /path/to/the/model \
3 --save_path /path/to/output/results \
4 --eval_set ... # {proofnet, connf}1@inproceedings{
2liu2025rethinking,
3title={Rethinking and improving autoformalization: towards a faithful metric and a Dependency Retrieval-based approach},
4author={Qi Liu and Xinhao Zheng and Xudong Lu and Qinxiang Cao and Junchi Yan},
5booktitle={The Thirteenth International Conference on Learning Representations},
6year={2025},
7url={https://openreview.net/forum?id=hUb2At2DsQ}
8}