You can deploy this model and start earning money today!

Ebtihal/
AraBertMo_base_V10

Name: AraBertMo_base_V10
Availability: InStock

Views

No views yet

Arabic Model AraBertMo_base_V10

language: ar tags: Fill-Mask datasets: OSCAR widget:

text: " السلام عليكم ورحمة[MASK] وبركاتة"
text: " اهلا وسهلا بكم في [MASK] من سيربح المليون"
text: " مرحبا بك عزيزي الزائر [MASK] موقعنا "

Arabic BERT Model

AraBERTMo is an Arabic pre-trained language model based on Google's BERT architechture. AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants All models are available on the HuggingFace model page under the Ebtihal name. Checkpoints are available in PyTorch formats.

Pretraining Corpus

`AraBertMo_base_V10' model was pre-trained on ~3 million words:

OSCAR - Arabic version "unshuffled_deduplicated_ar".

Training results

this model achieves the following results:

Task	Num examples	Num Epochs	Batch Size	steps	Wall time	training loss
Fill-Mask	30024	10	64	4700	9h 13m 43s	7.2395

Load Pretrained Model

You can use this model by installing torch or tensorflow and Huggingface library transformers. And you can use it directly by initializing it like this:

python

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("Ebtihal/AraBertMo_base_V10")
model = AutoModelForMaskedLM.from_pretrained("Ebtihal/AraBertMo_base_V10")

This model was built for master's degree research in an organization:

University of kufa.
Faculty of Computer Science and Mathematics.
Department of Computer Science