WSDM Cup 2023 BERT Checkpoints:

• This repo contains the checkpoints of our competition in WSDM Cup 2023: Pre-training for Web Search and Unbiased Learning for Web Search.

Paper released

• Task1 Unbiased Learning to rank: Multi-Feature Integration for Perception-Dependent Examination-Bias Estimation
• Task2 Pretraining for web search: Pretraining De-Biased Language Model with Large-scale Click Logs for Document Ranking

Method Overview

• Pre-training BERT with MLM and CTR prediction loss (or multi-task CTR prediction loss).
• Finetuning BERT with pairwise ranking loss.
• Obtain prediction scores from different BERTs.
• Ensemble learning to combine BERT features and sparse features.

BERT features:

1) Model details: Checkpoints Download Here

2) Method details

Contacts

• Xiangsheng Li: lixsh6@gmail.com.
• Xiaoshu Chen: xschenranker@gmail.com