-- This is the model checkpoint of FiD [2], based on the T5 (with 3B parameters) and trained on the WebQ dataset [1].

-- Hyperparameters: 8 x 40GB A100 GPUs; batch size 8; AdamW; LR 3e-5; 30000 steps

References:

[1] Semantic parsing on freebase from question-answer pairs. EMNLP 2013.

[2] Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. EACL 2021.

We evaluate it on the WebQ dataset, the EM score is 51.4 on the test set.