A domain-adapted BERT-base model, further pre-trained on the Sentiment140 dataset text.
This model is based on the
BERT base (uncased)
architecture and was further pre-trained (domain-adapted) using the text in Sentiment140 dataset, excluding its test split.
Only the masked language modeling (MLM) objective was used during domain adaptation.
Intermediate checkpoints from the pre-training process are available and can be accessed using specific tags,
which correspond to training epochs and steps:
1from transformers import AutoModelForMaskedLM
2
3model = AutoModelForMaskedLM.from_pretrained("<model-name>", revision="<checkpoint-tag>")
For more details on the training procedure, please refer to the base model's documentation:
Training procedure.
All texts from Sentiment140 dataset, excluding the test partition.
For typical use cases and limitations, please refer to the base model's guidance:
Inteded uses & limitations.
This model inherits potential risks and limitations from the base model. Refer to:
Limitations and bias.
