Views
No views yet
| Property | Value |
|---|---|
| Architecture | XLM-RoBERTa |
| Precision | 8-bit (affine quantization) |
| Embedding Dimension | 1024 |
| Max Sequence Length | 8192 |
| Model Size | ~592 MB |
| Quantization Group Size | 64 |
| Languages | 100+ languages |
| Version | Size | Compression |
|---|---|---|
| FP16 | 1.1 GB | - |
| 8-bit | 592 MB | 46% |
| 6-bit | 457 MB | 58% |
| 4-bit | 321 MB | 71% |
1from mlx_embeddings.utils import load_model, load_tokenizer
2import mlx.core as mx
3
4model_path = "mlx-community/bge-m3-mlx-8bit"
5
6# Load model and tokenizer
7model = load_model(model_path)
8tokenizer = load_tokenizer(model_path)
9
10# Generate embeddings
11text = "Hello, world!"
12tokens = tokenizer.encode(text)
13input_ids = mx.array([tokens])
14output = model(input_ids)
15embedding = output.last_hidden_state.mean(axis=1) # Mean pooling
16
17print(f"Embedding shape: {embedding.shape}") # (1, 1024)1curl http://127.0.0.1:8000/v1/embeddings \
2 -H "Content-Type: application/json" \
3 -d '{"model": "bge-m3-mlx-8bit", "input": "Your text here"}'1@article{bge_m3,
2 title={BGE M3-Embedding: Accurate, Efficient and Versatile Text Embedding},
3 author={Chen, Jianlv and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Zhang, Zheng},
4 journal={arXiv preprint arXiv:2402.03216},
5 year={2024}
6}