To reproduce the results, run the following command using the Lightning IR library and the configuration files from the repository repository linked above:

lightning-ir re_rank --config ./configs/re-rank.yaml --model.model_name_or_path <MODEL_NAME>

(nDCG@10 on TREC DL 19 and TREC DL 20)

Model Name	TREC DL 19 (BM25)	TREC DL 20 (BM25)	TREC DL 19 (ColBERTv2)	TREC DL 20 (ColBERTv2)
webis/set-encoder-base	0.746	0.704	0.781	0.768
webis/set-encoder-large	0.750	0.722	0.789	0.791

Citation

If you use this code or the models in your research, please cite our paper:

bibtex

@InProceedings{schlatt:2025,
  address =                  {Berlin Heidelberg New York},
  author =                   {Ferdinand Schlatt and Maik Fr{\"o}be and Harrisen Scells and Shengyao Zhuang and Bevan Koopman and Guido Zuccon and Benno Stein and Martin Potthast and Matthias Hagen},
  booktitle =                {Advances in Information Retrieval. 47th European Conference on IR Research (ECIR 2025)},
  doi =                      {10.1007/978-3-031-88711-6_1},
  month =                    apr,
  publisher =                {Springer},
  series =                   {Lecture Notes in Computer Science},
  site =                     {Lucca, Italy},
  title =                    {{Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders}},
  year =                     2025
}

webis/set-encoder-large

Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders

Citation

webis/
set-encoder-large