FaBERT is a Persian BERT-base model trained on the diverse HmBlogs corpus, encompassing both casual and formal Persian texts. Developed for natural language processing tasks, FaBERT is a robust solution for processing Persian text. Through evaluation across various Natural Language Understanding (NLU) tasks, FaBERT consistently demonstrates notable improvements, while having a compact model size. Now available on Hugging Face, integrating FaBERT into your projects is hassle-free. Experience enhanced performance without added complexity as FaBERT tackles a variety of NLP tasks.
1from transformers import AutoTokenizer, AutoModelForMaskedLM
2
3tokenizer = AutoTokenizer.from_pretrained("sbunlp/fabert") # make sure to use the default fast tokenizer
4model = AutoModelForMaskedLM.from_pretrained("sbunlp/fabert")
Similar to the original English BERT, FaBERT can be fine-tuned on many downstream tasks.(
https://huggingface.co/docs/transformers/en/training)
Examples on Persian datasets are available in our
GitHub repository.
FaBERT was pre-trained with the MLM (WWM) objective, and the resulting perplexity on validation set was 7.76.
For a more detailed performance analysis refer to the paper.
If you use FaBERT in your research or projects, please cite it using the following BibTeX:
1@inproceedings{masumi-etal-2025-fabert,
2 title = "{F}a{BERT}: Pre-training {BERT} on {P}ersian Blogs",
3 author = "Masumi, Mostafa and
4 Majd, Seyed Soroush and
5 Shamsfard, Mehrnoush and
6 Beigy, Hamid",
7 editor = "Bak, JinYeong and
8 Goot, Rob van der and
9 Jang, Hyeju and
10 Buaphet, Weerayut and
11 Ramponi, Alan and
12 Xu, Wei and
13 Ritter, Alan",
14 booktitle = "Proceedings of the Tenth Workshop on Noisy and User-generated Text",
15 month = may,
16 year = "2025",
17 address = "Albuquerque, New Mexico, USA",
18 publisher = "Association for Computational Linguistics",
19 url = "https://aclanthology.org/2025.wnut-1.10/",
20 doi = "10.18653/v1/2025.wnut-1.10",
21 pages = "85--96",
22 ISBN = "979-8-89176-232-9",
23}
