This model is a fine-tuned version of
gyeoldere/test_trainer on the snli dataset.
Test_trainer model is a fine-tuned version of
microsoft/deberta-base on the snli dataset.
This model fine-tuned to perform 2 tasks simultaneously; NLI task and MLM task.
Output vector of DeBERTa processed through two different fc layer to predict.
I used layer structure introduced in BERT paper, which is implemented on huggingface transformers; DebertaForTokenClassification and DebertaForMaskedLM.
[
https://huggingface.co/docs/transformers/index]
BinaryCrossEntrophyLoss are used for each class, and two losses are added to obtain final loss
final_loss = MLM_loss + NLI_loss
