First of all, we want to acknowledge the original creators of the
Snowflake/snowflake-arctic-embed-l and
BAAI/bge-base-en-v1.5 models which are used to create this model. Our model is just a combination of these two models, and we have not made any changes to the original models.
Furthermore, we want to acknowledge the team of Marqo, who has worked on the idea of combining two models through concatenation in parallel to ourselves. Their initial effort allowed to re-use existing pieces of code, in particular the
modeling script for bringing the combined model to HuggingFace.
Embedding models have become increasingly powerful and applicable across various use cases. However, the next significant challenge lies in enhancing their efficiency in terms of resource consumption. Our goal is to experiment with combining two embedding models to achieve better performance with fewer resources.
While the results are promising, we acknowledge the complexity of model combinations and the importance of focusing on more than leaderboard rankings. The simplicity of concatenating embeddings yielding tangible gains emphasizes the potential for further exploration in this area.
We look forward to conducting additional experiments and engaging in discussions to deepen our understanding of effective model combinations.
Please refer to our first concatenated model,
Arctic M (v1.5) & BGE small, for more details on how to use this model.
1@misc{https://doi.org/10.48550/arxiv.2407.08275,
2 doi = {10.48550/ARXIV.2407.08275},
3 url = {https://arxiv.org/abs/2407.08275},
4 author = {Caspari, Laura and Dastidar, Kanishka Ghosh and Zerhoudi, Saber and Mitrovic, Jelena and Granitzer, Michael},
5 title = {Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems},
6 year = {2024},
7 copyright = {Creative Commons Attribution 4.0 International}
8}
Notice that Arctic L is licensed under
Apache-2.0 and BGE Small (en; v1.5) is licensed under
MIT license. Please refer to the licenses of the original models for more details.
