BitNet-Embeddings is a family of multilingual text embedding models developed by Microsoft BitNet team.
The models use decoder-only architecture with last-token pooling and L2 normalization to produce dense text embeddings.
They can be applied to a wide range of tasks, including text retrieval, clustering, semantic similarity, classification, bitext mining, and reranking.
They achieve competitive performance on public benchmarks while maintaining excellent inference and storage efficiency.
The output embeddings can be quantized to 8, 4, 2, or even 1 bit, allowing users to flexibly trade off between storage cost and retrieval performance based on their application needs.
Embedding Quantization — Mean MTEB v2 Score
Training
The models are trained with contrastive learning objectives on a large-scale mixture of multilingual datasets covering diverse tasks.
Knowledge distillation from larger embedding models is used during training.
The BitNet quantization is applied to all linear layers, resulting in 1.58-bit ternary weights while keeping activations in higher precision.
How to Use (with bitnet.cpp)
For achieving the efficiency benefits, use the dedicated C++ implementation: bitnet.cpp.
Please refer to the bitnet.cpp GitHub repository for detailed compilation steps, usage examples, and command-line options.
Evaluation
Embedding Quality (MMTEB eng, v2)
BitNet Embedding 0.6B was evaluated against its full-precision FP16 teacher model on the MMTEB (eng, v2) benchmark:
Model
Cls.
Clust.
PairCls.
Rerank.
Retr.
STS
Summ.
Avg.
Speed (t/s)
FP16 Teacher
86.37
55.48
82.56
43.89
55.34
81.15
31.87
67.95
382.15
BitNet Embedding 0.6B
86.49
55.42
82.30
43.41
54.03
81.15
32.06
67.60
870.90
The model achieves 67.60 average score on MMTEB (eng, v2), only 0.35 points below the FP16 teacher, while delivering 2.28x higher CPU throughput.
Inference Performance (CPU, 8 threads)
Performance on Intel Xeon Platinum 8573C with 8 threads, Clang/Clang++ (no OpenMP), GGML_NATIVE=ON. All results in tokens/second (mean +/- std over 3 runs).
Note on build flags: The results below use -DGGML_NATIVE=ON, which auto-detects and enables the best instruction set supported by the host CPU (e.g., AVX, AVX2, AVX-VNNI, FMA, F16C). This yields optimal performance. To target only AVX2 (e.g., for portable binaries), set -DGGML_NATIVE=OFF and manually specify:
BitNet-Embeddings is designed for direct use in tasks requiring text embeddings. Its primary applications include:
Efficient information retrieval for RAG, web search, enterprise search, and question answering applications.
Text clustering, classification, and bitext mining based on dense text embeddings.
Out-of-Scope Use
BitNet-Embeddings does not generate any human-readable texts or other multi-modal content. Instead, it maps input texts into dense embedding vectors that are used by downstream Vector DB or classifiers.
Limited Training Data Representation: While the training data includes a mix of multilingual datasets, it may not represent diverse linguistic, cultural, or geographical contexts for low-resource languages. As a result, performance in low-resource languages may be significantly limited.
Domain-Specific Limitations: The training data predominantly covers general-purpose knowledge tasks. Specific or niche domains such as legal, medical, or scientific literature may not be adequately represented, and results in these areas should be interpreted with caution.
Use in High-Risk Applications: We do not recommend using BitNet-Embeddings in commercial or real-world applications without further testing and development. As the model is being released for research purposes, additional evaluation and fine-tuning are required to ensure reliability and fairness in high-stakes scenarios.
Supported Languages
The models are trained on multilingual data and support a wide range of languages,
including but not limited to: Arabic, Bulgarian, Catalan, Czech, Danish, German, Greek, English, Spanish,
Estonian, Persian, Finnish, French, Hebrew, Hindi, Croatian, Hungarian, Indonesian, Italian, Japanese,
Korean, Lithuanian, Latvian, Macedonian, Malay, Dutch, Norwegian, Polish, Portuguese, Romanian, Russian,
Slovak, Slovenian, Albanian, Serbian, Swedish, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Chinese.
Evaluation
Please follow the mteb repository on how to reproduce our scores.
The evaluation prompts used for each task are also available at mteb_v2_eval_prompts.json.
Bias, Risks, and Limitations
Predictions may perpetuate biases present in the training data.
There is limited support for extremely low-resource languages and underrepresented domains.
The model is designed for text embedding tasks (retrieval, similarity, classification) and is not suitable for text generation.
Disclaimer
We do not recommend using BitNet Embedding 0.6B in commercial or real-world applications without further testing and development. This model is intended for research and development purposes. Please use responsibly.
FAQ
1. Do I need to add instructions to the query?
Yes, this is how the model is trained, otherwise you will see a performance degradation.
The task definition should be a one-sentence instruction that describes the task.
This is a way to customize text embeddings for different scenarios through natural language instructions.
On the other hand, there is no need to add instructions to the document side.
2. Why are my reproduced results slightly different from reported in the model card?
Different versions of transformers and pytorch could cause negligible but non-zero performance differences.
3. What pooling strategy does this model use?
The model uses last-token pooling — the embedding of the last non-padding token is used as the sentence representation.
The embedding is then L2-normalized.
Citation
bibtex
1@misc{li2026bitnettextembeddings,
2 title={BitNet Text Embeddings},
3 author={Zhen Li and Xin Huang and Liang Wang and Nan Yang and Ting Song and Yan Xia and Xun Wu and Shaohan Huang and Huishuai Zhang and Furu Wei and Dongyan Zhao},
4 year={2026},
5 eprint={2606.25674},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2606.25674}
9}
1011@inproceedings{wang2025bitnet,
12 title={BitNet.cpp: Efficient Edge Inference for Ternary LLMs},
13 author={Wang, Jinheng and Zhou, Hansong and Song, Ting and Cao, Shijie and Xia, Yan and Cao, Ting and Wei, Jianyu and Ma, Shuming and Wang, Hongyu and Wei, Furu},
14 booktitle={Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
15 pages={9305--9322},
16 year={2025}
17}
1819@article{wang2024multilingual,
20 title={Multilingual E5 Text Embeddings: A Technical Report},
21 author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
22 journal={arXiv preprint arXiv:2402.05672},
23 year={2024}
24}
