Objective:
This is Roberta Base with Domain Adaptive Pretraining on Movie Corpora --> Then a changed head to do the SQuAD Task. This makes a QA model capable of answering questions in the movie domain. https://huggingface.co/thatdramebaazguy/movie-roberta-base was used as the MovieRoberta.
Language model: roberta-base Language: English Downstream-task: QA Training data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names, SQuADv1 Eval data: MoviesQA (From https://github.com/ibm-aur-nlp/domain-specific-QA) Infrastructure: 1x Tesla v100 Code: See example
Hyperparameters
Num examples = 88567
Num Epochs = 10
Instantaneous batch size per device = 32
Total train batch size (w. parallel, distributed & accumulation) = 32
