Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in
this paper and first released in
this repository. This model is uncased: it does not make a difference
between english and English.
You can use the raw model for masked language modeling or next sentence prediction, but it is mostly intended to be fine-tuned on a downstream task. It is primarily aimed at tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification, or question answering.
As noted in the upstream model card, even if the training data used for this model could be characterized as fairly neutral, this model can have biased predictions. This bias will also affect all fine-tuned versions of this model. The model was not fine-tuned for production use.
This model is licensed under the Apache-2.0 license.
