NeoAraBERT is a state-of-the-art open-source Arabic text-embedding model built on the NeoBERT architecture. This project was a collaboration between the Arab Center for Research and Policy Studies’ (ACRPS) Unit for Research In Arabic Social and Digital Spaces (U4RASD) and the American University of Beirut (AUB).
We pretrain NeoAraBERT on diverse open-source and internal datasets covering modern standard, classical, and dialectal Arabic. We guided our design choices with Arabic tailored ablation studies including text normalization, light stemming, and diacritics-aware tokenization handling. We also performed POS-aware token masking and learning-rate scheduling ablation studies. We benchmarked NeoAraBERT against five top-performing Arabic models on 23 tasks, including a synonym-based task,
Muradif, that directly assesses embedding quality with no additional fine-tuning. NeoAraBERT variants rank first in 18 tasks and improve average performance across the full benchmark suite.
For detailed benchmarking, see
https://acr.ps/neoarabert.
1from transformers import AutoModel, AutoTokenizer
2
3model_name = "U4RASD/NeoAraBERT_MSA"
4tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
5model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
6
7# Tokenize input text
8text = "المركز العربيّ للأبحاث ودراسة السياسات."
9inputs = tokenizer(text, return_tensors="pt")
10
11# Generate embeddings
12outputs = model(**inputs)
13embedding = outputs.last_hidden_state[:, 0, :]
14print(embedding.shape)
1@inproceedings{abou-chakra-etal-2026-neoarabert,
2 title = "{N}eo{A}ra{BERT}: A Modern Foundation Model for {A}rabic Embeddings with Diacritics-Aware Tokenization and {POS}-Targeted Masking",
3 author = "Abou Chakra, Chadi and
4 Hamoud, Hadi Khaled and
5 Rakan Al Mraikhat, Osama and
6 Abu Obaida, Qusai and
7 Ballout, Mohamad and
8 Zaraket, Fadi",
9 editor = "Liakata, Maria and
10 Moreira, Viviane P. and
11 Zhang, Jiajun and
12 Jurgens, David",
13 booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
14 month = jul,
15 year = "2026",
16 address = "San Diego, California, United States",
17 publisher = "Association for Computational Linguistics",
18 url = "https://aclanthology.org/2026.findings-acl.1293/",
19 doi = "10.18653/v1/2026.findings-acl.1293",
20 pages = "25952--25968",
21 ISBN = "979-8-89176-395-1",
22 abstract = "We present NeoAraBERT, a state-of-the-art open-source Arabic text-embedding model built on the NeoBERT architecture. We pre-train NeoAraBERT on diverse open-source and internal datasets covering modern standard, classical, and dialectal Arabic. We guided our design choices with Arabic tailored ablation studies including text normalization, light stemming, and diacritics-aware tokenization handling. We also performed more general POS-aware token masking and learning-rate scheduling ablation studies. We benchmarked NeoAraBERT against five top-performing Arabic models on 23 tasks, including a novel synonym-based task, ``Muradif'', that directly assesses embedding quality with no additional fine-tuning. NeoAraBERT variants (MSA, dialectal, and mixed) rank first in 18 tasks, second in two, third in two, and fourth in one task. They show strong performance on classical and modern standard Arabic, substantial margins of improvement ($>$7{\%}) in two tasks, and a $+$2.75{\%} improvement on average across all tasks. Our code and links to checkpoints for our model variants are available on our website: \url{https://acr.ps/neoarabert}."
23}
We would like to acknowledge Ahmad Talal Salman from Assafir and Professor Amer Abdo Mouawad from the American University of Beirut for sharing Assafir data, which was instrumental to the work presented in this paper.
This model is licensed under the CC BY-SA 4.0 license. The text of the license can be found
here.
