This Arabic ColBERT model is reasonably, but not fully, trained on 250k normalized queries sampled from the Arabic mMARCO dataset.
Training parameters are in the metadata file.
See
https://www.linkedin.com/posts/akhooli_arabic-bert-tokenizers-you-may-need-to-normalize-activity-7225747473523216384-D1oH
Please note that there is another model trained (partially) on normalized 711k
dataset:
akhooli/arabic-colbertv2-711k-norm.
This model should be good for ranking and retrieval but not for critical tasks. A demo example using it is the
Quran Semantic Search.
If you downloaded it before Aug. 6, 2024, you are advised to refresh your copy.
1from unicodedata import normalize
2query_n = normalize('NFKC', query)
