This is bart-large model finetuned on SQuADv1 dataset for question answering task
Model details
BART was propsed in the paperBART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.
BART is a seq2seq model intended for both NLG and NLU tasks.
To use BART for question answering tasks, we feed the complete document into the encoder and decoder, and use the top
hidden state of the decoder as a representation for each
word. This representation is used to classify the token. As given in the paper bart-large achives comparable to ROBERTa on SQuAD.
Another notable thing about BART is that it can handle sequences with upto 1024 tokens.
Param
#Value
encoder layers
12
decoder layers
12
hidden size
4096
num attetion heads
16
on disk size
1.63GB
Model training
This model was trained on google colab v100 GPU.
You can find the fine-tuning colab here
