mixedbread-ai/mxbai-embed-xsmall-v1

This model is an open-source English embedding model developed by Mixedbread. It's built upon sentence-transformers/all-MiniLM-L6-v2 and trained with the AnglE loss and Espresso. Read more details in our blog post.

In a bread loaf:

State-of-the-art performance

Supports both binary quantization and Matryoshka Representation Learning (MRL).

Optimized for retrieval tasks

4096 context support

Performance

Binary Quantization and Matryoshka

Our model supports both binary quantization and Matryoshka Representation Learning (MRL), allowing for significant efficiency gains:

Binary quantization: Retains 93.9% of performance while increasing efficiency by a factor of 32

MRL: A 33% reduction in vector size still leaves 96.2% of model performance

These optimizations can lead to substantial reductions in infrastructure costs for cloud computing and vector databases. Read more here.

Quickstart

Here are several ways to produce German sentence embeddings using our model.

angle-emb

pip install -U angle-emb

python
1from angle_emb import AnglE 2from angle_emb.utils import cosine_similarity 3 4# 1. Specify preferred dimensions 5dimensions = 384 6 7# 2. Load model and set pooling strategy to avg 8model = AnglE.from_pretrained( 9 "mixedbread-ai/mxbai-embed-xsmall-v1", 10 pooling_strategy='avg').cuda() 11 12query = 'A man is eating a piece of bread' 13 14docs = [ 15 query, 16 "A man is eating food.", 17 "A man is eating pasta.", 18 "The girl is carrying a baby.", 19 "A man is riding a horse.", 20] 21 22# 3. Encode 23embeddings = model.encode(docs, embedding_size=dimensions) 24 25for doc, emb in zip(docs[1:], embeddings[1:]): 26 print(f'{query} ||| {doc}', cosine_similarity(embeddings[0], emb))

Sentence Transformers

python -m pip install -U sentence-transformers

python
1from sentence_transformers import SentenceTransformer 2from sentence_transformers.util import cos_sim 3 4# 1. Specify preferred dimensions 5dimensions = 384 6 7# 2. Load model 8model = SentenceTransformer("mixedbread-ai/mxbai-embed-xsmall-v1", truncate_dim=dimensions) 9 10query = 'A man is eating a piece of bread' 11 12docs = [ 13 query, 14 "A man is eating food.", 15 "A man is eating pasta.", 16 "The girl is carrying a baby.", 17 "A man is riding a horse.", 18] 19 20 21# 3. Encode 22embeddings = model.encode(docs) 23 24similarities = cos_sim(embeddings[0], embeddings[1:]) 25print('similarities:', similarities)

transformers

pip install -U transformers

python
1from typing import Dict 2 3import torch 4import numpy as np 5from transformers import AutoModel, AutoTokenizer 6from sentence_transformers.util import cos_sim 7 8def pooling(outputs: torch.Tensor, inputs: Dict) -> np.ndarray: 9 outputs = torch.sum( 10 outputs * inputs["attention_mask"][:, :, None], dim=1) / torch.sum(inputs["attention_mask"]) 11 return outputs.detach().cpu().numpy() 12 13# 1. Load model 14model_id = 'mixedbread-ai/mxbai-embed-xsmall-v1' 15tokenizer = AutoTokenizer.from_pretrained(model_id) 16model = AutoModel.from_pretrained(model_id).cuda() 17 18query = 'A man is eating a piece of bread' 19 20docs = [ 21 query, 22 "A man is eating food.", 23 "A man is eating pasta.", 24 "The girl is carrying a baby.", 25 "A man is riding a horse.", 26] 27 28# 2. Encode 29inputs = tokenizer(docs, padding=True, return_tensors='pt') 30for k, v in inputs.items(): 31 inputs[k] = v.cuda() 32outputs = model(**inputs).last_hidden_state 33embeddings = pooling(outputs, inputs) 34 35# 3. Compute similarity scores 36similarities = cos_sim(embeddings[0], embeddings[1:]) 37print('similarities:', similarities)

Batched API

python -m pip install batched

python
1import uvicorn 2import batched 3from fastapi import FastAPI 4from fastapi.responses import ORJSONResponse 5from sentence_transformers import SentenceTransformer 6from pydantic import BaseModel 7 8app = FastAPI() 9 10model = SentenceTransformer('mixedbread-ai/mxbai-embed-xsmall-v1') 11model.encode = batched.aio.dynamically(model.encode) 12 13class EmbeddingsRequest(BaseModel): 14 input: str | list[str] 15 16@app.post("/embeddings") 17async def embeddings(request: EmbeddingsRequest): 18 return ORJSONResponse({"embeddings": await model.encode(request.input)}) 19 20if __name__ == "__main__": 21 uvicorn.run(app, host="0.0.0.0", port=8000)

Community

Join our discord community to share your feedback and thoughts. We're here to help and always happy to discuss the exciting field of machine learning!

License

Apache 2.0

Citation

bibtex
1@online{xsmall2024mxbai, 2 title={Every Byte Matters: Introducing mxbai-embed-xsmall-v1}, 3 author={Sean Lee and Julius Lipp and Rui Huang and Darius Koenig}, 4 year={2024}, 5 url={https://www.mixedbread.ai/blog/mxbai-embed-xsmall-v1}, 6}