Zarra is a static embedding model built using the Model2Vec distillation framework.
It is a distilled version of a Sentence Transformer, specifically optimized for the Arabic language.
Unlike traditional transformer-based models, Zarra produces static embeddings, enabling ultra-fast inference on both CPU and GPU—making it ideal for resource-constrained environments or real-time applications.
Why Zarra?
⚡ Exceptional Speed: Delivers embeddings up to 500x faster than sentence transformers.
🧠 Compact & Efficient: Up to 50x smaller in size, allowing easy deployment on edge devices.
🧰 Versatile: Well-suited for search, clustering, classification, deduplication, and more.
🌍 Arabic-First: Specifically trained on high-quality Arabic data, ensuring relevance and performance across a range of Arabic NLP tasks.
Speed vs Performance Chart
About Model2Vec
The Model2Vec distillation technique transfers knowledge from large transformer models into lightweight static embedding spaces, preserving semantic quality while dramatically improving speed and efficiency.
Zarra represents the best of both worlds: the semantic power of transformers and the speed and simplicity of static vectors.
Installation
Install model2vec using pip:
pip install model2vec
Usage
Using Model2Vec
The Model2Vec library is the fastest and most lightweight way to run Model2Vec models.
Load this model using the from_pretrained method:
python
1from model2vec import StaticModel
23# Load a pretrained Model2Vec model4model = StaticModel.from_pretrained("NAMAA-Space/zarra")56# Compute text embeddings7embeddings = model.encode(["Example sentence"])
1from sentence_transformers import SentenceTransformer
23# Load a pretrained Sentence Transformer model4model = SentenceTransformer("NAMAA-Space/zarra")56# Compute text embeddings7embeddings = model.encode(["Example sentence"])
How it Works
Model2vec creates a small, fast, and powerful model that outperforms other static embedding models by a large margin on all tasks we could find, while being much faster to create than traditional static embedding models such as GloVe. Best of all, you don't need any data to distill a model using Model2Vec.
It works by passing a vocabulary through a sentence transformer model, then reducing the dimensionality of the resulting embeddings using PCA, and finally weighting the embeddings using SIF weighting. During inference, we simply take the mean of all token embeddings occurring in a sentence.
Benchmark on Arabic
Speed
Model
Speed (sentences/second)
Device
zarra
26893.63
cpu
bojji
27478.15
cpu
potion-multilingual-128M
27145.31
cpu
paraphrase-multilingual-MiniLM-L12-v2
2363.24
cuda
silma_ai_embedding_sts_v0.1
627.13
cuda
muffakir_embedding
621.77
cuda
get_multilingual_base
895.41
cuda
arabic_retrieval_v1.0
618.56
cuda
arabic_triplet_matryoshka_v2
610.64
cuda
Zarra and Bojji excel in speed, achieving 26893.63 and 27478.15 sentences per second on CPU, respectively, far surpassing CUDA-based models like arabic_triplet_matryoshka_v2 (610.64).
Top Performer: Bojji is the fastest model, slightly ahead of Zarra and potion-multilingual-128M (27145.31), highlighting the efficiency of Model2Vec-based models on CPU.
Key Observation: The high speed of Zarra and Bojji on CPU makes them ideal for resource-constrained environments, offering significant advantages over CUDA-dependent models.
Size of the Model
Model
Parameters (M)
Size (MB)
Relative to Largest (%)
Less than Largest (x)
zarra
64.00
244.14
41.92
2.39
bojji
124.88
476.40
81.79
1.22
potion-multilingual-128M
128.09
488.63
83.89
1.19
paraphrase-multilingual-MiniLM-…
117.65
448.82
77.06
1.30
silma_ai_embedding_sts_v0.1
135.19
515.72
88.54
1.13
muffakir_embedding
135.19
515.72
88.54
1.13
arabic_retrieval_v1.0
135.19
515.73
88.54
1.13
arabic_triplet_matryoshka_v2
135.19
515.72
88.54
1.13
get_multilingual_base
305.37
582.45
100.00
1.00
Zarra is the smallest model, with only 64 million parameters and 244.14 MB in size, making it 2.39 times smaller than the largest model (get_multilingual_base).
Bojji is slightly larger at 124.88 million parameters and 476.40 MB, but still significantly smaller than most other models.
Top Performer: Zarra leads in compactness, offering the smallest footprint, which is critical for deployment on resource-limited devices.
Key Observation: The compact size of Zarra and Bojji aligns with their design goal of efficiency, making them highly suitable for edge computing and real-time applications.
