ESG analysis can help investors determine a business' long-term sustainability and identify associated risks. MaskESG-DistilBERT-base is a distilbert/distilbert-base-multilingual-cased model fine-tuned on EnVi-ESG-200 dataset, include 200,000 annotated sentences from Vietnam, English news and ESG reports.

Input: A financial text.

Output: Environmental, Social, Governance or Neural.

Language support: English, Vietnamese

How to use

You can use this model with Transformers pipeline for ESG classification or fill mask task.

python

# tested in transformers==4.53.0
from transformers import AutoTokenizer, AutoModelForMaskedLM, pipeline

maskesg = AutoModelForMaskedLM.from_pretrained('nguyen599/MaskESG-DistilBERT-base')
tokenizer = AutoTokenizer.from_pretrained('nguyen599/MaskESG-DistilBERT-base')
nlp = pipeline("fill-mask", model=maskesg, tokenizer=tokenizer)
# Classification as fill-mask
results = nlp(f'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is {tokenizer.mask_token}')
print(results)
# [{'score': 0.9015821814537048,
#   'token': 444,
#   'token_str': ' E',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is E'},
#  {'score': 0.09723947197198868,
#   'token': 427,
#   'token_str': ' N',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is N'},
#  {'score': 0.0010556845227256417,
#   'token': 322,
#   'token_str': ' S',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is S'},
#  {'score': 0.0001152529803221114,
#   'token': 443,
#   'token_str': ' G',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is G'},
#  {'score': 1.14425779429439e-06,
#   'token': 299,
#   'token_str': ' e',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is e'}]