The EnergyBERT model can be expanded way beyond just text classification. It can be fine-tuned to perform various other downstream NLP tasks in the domain of Energy & Material
Use the code below to get started with the model.
1from transformers import pipeline
2unmasker = pipeline('fill-mask', model='EnergyBERT')
3unmasker("Hello I'm a <mask> model.")
1.2M Published full-text literature corpus from 2000 to 2021.
BERT is trained on two unsupervised tasks during its pre-training period: masked language modeling and next sentence prediction. A masked language model involves masking some of the input tokens at random and training the model to predict the masked tokens based on the context surrounding the input tokens. Next sentence prediction involves training the model to predict whether two sentences follow each other logically.
