Views
No views yet
AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages which will appear at the Third Simple and Efficient Natural Language Processing, at EMNLP 2022.
| Model | MasakhaNER | MasakhaNER2.0* | Text Classification (Yoruba/Hausa) | Sentiment Analysis (YOSM) | OOD Sentiment Analysis (Twitter -> YOSM) |
|---|---|---|---|---|---|
AfroLM-Large | 80.13 | 83.26 | 82.90/91.00 | 85.40 | 68.70 |
AfriBERTa | 79.10 | 81.31 | 83.22/90.86 | 82.70 | 65.90 |
mBERT | 71.55 | 80.68 | --- | --- | --- |
XLMR-base | 79.16 | 83.09 | --- | --- | --- |
AfroXLMR-base | 81.90 | 84.55 | --- | --- | --- |
1from transformers import XLMRobertaModel, XLMRobertaTokenizer
2model = XLMRobertaModel.from_pretrained("bonadossou/afrolm_active_learning")
3tokenizer = XLMRobertaTokenizer.from_pretrained("bonadossou/afrolm_active_learning")
4tokenizer.model_max_length = 256Autotokenizer class does not successfully load our tokenizer. So we recommend to use directly the XLMRobertaTokenizer class. Depending on your task, you will load the according mode of the model. Read the XLMRoberta Documentationpython active_learning.py. You can also wrap it around a bash script.bash ner_experiments.shbash text_classification_all.sh@inproceedings{dossou-etal-2022-afrolm, title = "{A}fro{LM}: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 {A}frican Languages", author = "Dossou, Bonaventure F. P. and Tonja, Atnafu Lambebo and Yousuf, Oreen and Osei, Salomey and Oppong, Abigail and Shode, Iyanuoluwa and Awoyomi, Oluwabusayo Olufunke and Emezue, Chris", booktitle = "Proceedings of The Third Workshop on Simple and Efficient Natural Language Processing (SustaiNLP)", month = dec, year = "2022", address = "Abu Dhabi, United Arab Emirates (Hybrid)", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.sustainlp-1.11", pages = "52--64"}