KaLM-embedding-multilingual-mini-instruct-v2.5 (ONNX)

Name: KaLM-embedding-multilingual-mini-instruct-v2.5-ONNX
Availability: InStock

This is an ONNX version of KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5. It was automatically converted and uploaded using this Hugging Face Space.

Usage with Transformers.js

See the pipeline documentation for feature-extraction: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.FeatureExtractionPipeline

KaLM-Embedding-V2.5

HuggingFace

GitHub Code

Paper

Short Description

KaLM-Embedding-V2.5 is a versatile and compact embedding model, which achieves SOTA performance among models of comparable size and competes with models 3–26x larger by leveraging superior training techniques and data.

Model Details

Model Size: 0.5B
Embedding Dimension: 896
Max Input Tokens: 32k
MRL dimensions: 896, 512, 256, 128, and 64
Attn: Bidirectional attention
Pooling: Mean pooling

Training Recipe

Large-scale weakly supervised pretraining
High-quality supervised finetuning
Contrastive distillation with fine-grained soft labels

Additionally, focal-style sample reweighting and online hard-negative mixing are employed to emphasize difficult samples and enrich hard negatives.

📑 Open-source Plan

Model Checkpoint
Training and Evaluation Code: HITsz-TMG/KaLM-Embedding
Technical Report: KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model
Pre-training Data: Pre-training Data
Fine-tuning Data: Fine-tuning Data

Evaluation

Overall results on MTEB (cmn, v1) and MTEB (eng, v1).

Detailed model performance on MTEB (cmn, v1).

Detailed model performance on MTEB (eng, v1).

OOD evaluation: KaLM-Embedding-V2.5 exhibits strong OOD generalization, competing with the 15x larger model in real-world retrieval scenarios.

Matryoshka embedding evaluation: KaLM-Embedding-V2.5 maintains robust performance with matryoshka embeddings even at smaller dimensions.

Requirements

Since we have used the Qwen2 model, we advise you to install transformers>=4.37.0, or you might encounter the following error:

KeyError: 'qwen2'

Usage

sentence-transformers support

Using this model becomes easy when you have sentence-transformers installed:

pip install -U sentence-transformers

Then you can use the model like this:

python

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer(
    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
    trust_remote_code=True,
    model_kwargs={
        "torch_dtype": torch.bfloat16,
        "attn_implementation": "flash_attention_2",  # Optional
    },
)
model.max_seq_length = 512

sentences = ["This is an example sentence", "Each sentence is converted"]
embeddings = model.encode(
    sentences,
    normalize_embeddings=True,
    batch_size=256,
    show_progress_bar=True,
)
print(embeddings)
'''
[[-0.01043701 -0.02172852  0.0100708  ... -0.02807617  0.00157166
  -0.03637695]
 [-0.00424194  0.02966309  0.03686523 ... -0.02587891  0.01953125
  -0.00125122]]
'''

We add task instructions for asymmetric tasks: retrieval, reranking, classification, and clustering. And, we add task instructions for both queries and passages in symmetric tasks, including STS and pair classification. If you want to add task instructions to the query, you can use the model like this:

python

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer(
    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
    trust_remote_code=True,
    model_kwargs={
        "torch_dtype": torch.bfloat16,
        "attn_implementation": "flash_attention_2",  # Optional
    },
)
model.max_seq_length = 512

sentences = ["This is an example sentence", "Each sentence is converted"]
prompt = "Instruct: Classifying the category of french news.\nQuery:"
embeddings = model.encode(
    sentences,
    prompt=prompt,
    normalize_embeddings=True,
    batch_size=256,
    show_progress_bar=True,
)
print(embeddings)
'''
[[-0.01867676  0.02319336  0.00280762 ... -0.02075195  0.00196838
  -0.0703125 ]
 [-0.0067749   0.03491211  0.01434326 ... -0.0043335   0.00509644
  -0.04174805]]
'''

Or you can use encode_query and encode_document to automatically add the default prompt for queries ("Instruct: Given a query, retrieve documents that answer the query \n Query: ") and documents (""), respectively.

python

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer(
    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
    trust_remote_code=True,
    model_kwargs={
        "torch_dtype": torch.bfloat16,
        "attn_implementation": "flash_attention_2",  # Optional
    },
)
model.max_seq_length = 512

queries = [
    "What is the capital of China?",
    "Explain gravity",
]
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)

similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
'''
tensor([[0.9034, 0.2563],
        [0.3153, 0.7396]])
'''

vllm support

pip install -U vllm==0.8.5

python

import torch
import vllm
from vllm import LLM
def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery:{query}'

task = 'Given a query, retrieve documents that answer the query'
queries = [
    get_detailed_instruct(task, 'What is the capital of China?'),
    get_detailed_instruct(task, 'Explain gravity')
]
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
]
input_texts = queries + documents

model = LLM(model="{MODEL_NAME_OR_PATH}", task="embed", trust_remote_code=True, dtype="float16")

outputs = model.embed(input_texts)
embeddings = torch.tensor([o.outputs.embedding for o in outputs])
scores = (embeddings[:2] @ embeddings[2:].T)
print(scores.tolist())

Citation

If you find this model useful, please consider giving a star and citation.

@misc{zhao2025kalmembeddingv2,
      title={KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model}, 
      author={Xinping Zhao and Xinshuo Hu and Zifei Shan and Shouzheng Huang and Yao Zhou and Xin Zhang and Zetian Sun and Zhenyu Liu and Dongfang Li and Xinyuan Wei and Youcheng Pan and Yang Xiang and Meishan Zhang and Haofen Wang and Jun Yu and Baotian Hu and Min Zhang},
      year={2025},
      eprint={2506.20923},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.20923}, 
}

@misc{hu2025kalmembedding,
      title={KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model}, 
      author={Xinshuo Hu and Zifei Shan and Xinping Zhao and Zetian Sun and Zhenyu Liu and Dongfang Li and Shaolin Ye and Xinyuan Wei and Qian Chen and Baotian Hu and Haofen Wang and Jun Yu and Min Zhang},
      year={2025},
      eprint={2501.01028},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.01028}, 
}

Contact

If you encounter any issue, feel free to contact us via the email: xinpingzhao@slai.edu.cn, yanshek.woo@gmail.com

thomasht86/KaLM-embedding-multilingual-mini-instruct-v2.5-ONNX

KaLM-embedding-multilingual-mini-instruct-v2.5 (ONNX)

Usage with Transformers.js

KaLM-Embedding-V2.5

Short Description

Model Details

Training Recipe

📑 Open-source Plan

Evaluation

Overall results on MTEB (cmn, v1) and MTEB (eng, v1).

Detailed model performance on MTEB (cmn, v1).

Detailed model performance on MTEB (eng, v1).

OOD evaluation: KaLM-Embedding-V2.5 exhibits strong OOD generalization, competing with the 15x larger model in real-world retrieval scenarios.

Matryoshka embedding evaluation: KaLM-Embedding-V2.5 maintains robust performance with matryoshka embeddings even at smaller dimensions.

Requirements

Usage

sentence-transformers support

vllm support

Citation

Contact

thomasht86/
KaLM-embedding-multilingual-mini-instruct-v2.5-ONNX