This repository contains the Step 2 (SFT) checkpoint used as the reference policy (π_ref) / initialization for the DDRO training stage from Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval.

What this is

Stage: Step 2 / supervised fine-tuning (SFT)
Role in DDRO: reference policy (π_ref)

Corresponding final DDRO model

Post-DDRO (Step 3) checkpoint: kiyam/ddro-nq-tu

How to load

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo = "kiyam/ddro-nq-tu-sft"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

Intended use

Research and reproducibility. Please cite the DDRO paper if you use this checkpoint.