Knowledge Distillation from strong synthetic data (200k samples generated with Qwen/Qwen3-32B-AWQ and scored by a high-performing reranker). Extreme compression with just 15M parameters – optimized for edge deployment and resource-constrained environments.

🎯 Core Features and Innovations:

Next-Generation Knowledge Distillation: By utilizing 200,000 synthetically generated, high-quality training examples (created with Qwen/Qwen3-32B-AWQ and scored by a state-of-the-art reranker).
Ultra-Efficient Architecture: With extreme parameter compression to just 15M, enabling deployment on edge devices and mobile platforms

💪 The Ant Among Giants: Tiny but Effective

With 15 million parameters – that's less than 1/500th the size of some competing models – SauerkrautLM-Reason-Multi-ColBERT-15m represents the extreme frontier of model compression:

500× smaller than 7B+ parameter models
10× smaller than typical BERT models
Comparable to SBERT-scale encoders in size

What makes SauerkrautLM-Multi-ColBERT-15m truly remarkable is not just its tiny size, but its exceptional performance in scientific and technical domains:

Domain-Specific Strengths:

Scientific Retrieval Champion: In Biology and Earth Sciences, it achieves scores (23.33, 23.78) that surpass E5 (1B parameters) and proprietary APIs
Technical Q&A Powerhouse: Mean StackExchange score of 14.64 puts it in the same league as models 10-100× larger
🐴 PONY TASK SUPERSTAR: Ranks #2 globally with 15.86, only behind GritLM-7B (22.00), crushing ReasonIR-8B (10.50) and Qwen-7B (9.90) - a 15M model beating 8 BILLION parameter models!
STEM-Optimized: The knowledge distillation process has successfully preserved critical scientific reasoning capabilities

Bright Performance Comparisons:

Domain	SauerkrautLM-15M	BGE	E5 (1B)	ReasonIR-8B	Qwen-7B	Performance
Biology	23.33	11.70	18.60	26.20	30.60	2× BGE, 125% of E5
Earth Science	23.78	24.60	26.00	31.40	36.40	97% of BGE, 91% of E5
Mean StackEx	14.64	15.17	17.36	24.76	22.80	96% of BGE, 84% of E5
🐴 Pony	15.86	5.70	4.90	10.50	9.90	#2 Overall! 278% of BGE, 151% of 8B model, 160% of 7B!

Model Overview

Model: VAGOsolutions/SauerkrautLM-Reason-Multi-ColBERT-15m
Base: Fine-tuned from VAGOsolutions/SauerkrautLM-Multi-ColBERT-15m using knowledge distillation
Architecture: PyLate / ColBERT (Late Interaction)
Languages: Multilingual (optimized for 7 European languages: German, English, Spanish, French, Italian, Dutch, Portuguese)
License: Apache 2.0
Model Size: 15M parameters Efficiency Ratio: Up to 500× smaller than comparable performing models

Model Description

Model Type: PyLate model with innovative Late Interaction architecture
Document Length: 2048 tokens (8× longer than traditional BERT models)
Query Length: 256 tokens (optimized for complex, multi-part queries)
Output Dimensionality: 128 tokens (efficient vector representation)
Similarity Function: MaxSim (enables precise token-level matching)
Training Loss: Knowledge Distillation (PyLate) - learns from models 2000× larger

Architecture

ColBERT(
  (0): Transformer(CompressedModernBertModel)
  (1): Dense(384 -> 128 dim, no bias)
)

🔬 Technical Innovations in Detail

Knowledge Distillation: David Learning from Goliath

Our 15M parameter model leverages state-of-the-art knowledge distillation:

Synthetic Data Generation: 200,000 high-quality query-document pairs generated using the Qwen/Qwen3-32B-AWQ model (32 billion parameters) based on the ReasonIR approach
Quality Assurance: Each pair evaluated and filtered by a state-of-the-art reranker

Ultra-Compact Design

SauerkrautLM-Reason-Multi-ColBERT-15m achieves extreme efficiency through:

Ultra-Compact Architecture (~15 M params)
Deep-yet-slim BERT — 10 layers, hidden_size = 288.
Many heads — 12 attention heads (24-dim each) for fine-grained reasoning in a narrow model.
Edge-ready — small footprint

This architecture enables Late Interaction Retrieval on mobile devices (or on even calculators :D).

🔬 Benchmarks: The Ant's Performance

Despite its microscopic size, SauerkrautLM-Reason-Multi-ColBERT-15m delivers functional retrieval performance, particularly impressive given its 500× size disadvantage compared to state-of-the-art models.

BRIGHT Benchmark (English, reasoning‑focused retrieval)

The BRIGHT benchmark is designed to evaluate reasoning‑intensive retrieval. All scores are nDCG@10. SauerkrautLM-Reason-Multi-ColBERT (15M parameters) demonstrates that extreme compression is possible while maintaining basic retrieval functionality.

Model / Metric	Biology	Earth	Economics	Psychology	Robotics	Stackoverflow	Sustainable	Leetcode	Pony	AoPS	Theorem‑Q	Theorem‑T	Mean StackEx	Mean coding	Mean theorem	Full Mean
BM25	18.90	27.20	14.90	12.50	13.60	18.40	15.00	24.40	7.90	6.20	10.40	4.90	17.21	16.15	7.17	14.53
< 1 B OS
BGE	11.70	24.60	16.60	17.50	11.70	10.80	13.30	26.70	5.70	6.00	13.00	6.90	15.17	16.20	8.63	13.71
Inst‑L	15.20	21.20	14.70	22.30	11.40	13.30	13.50	19.50	1.30	8.10	20.90	9.10	15.94	10.40	12.70	14.21
SBERT	15.10	20.40	16.60	22.70	8.20	11.00	15.30	26.40	7.00	5.30	20.00	10.80	15.61	16.70	12.03	14.90
> 1 B OS
E5	18.60	26.00	15.50	15.80	16.30	11.20	18.10	28.70	4.90	7.10	26.10	26.80	17.36	16.80	20.00	17.93
SFR	19.10	26.70	17.80	19.00	16.30	14.40	19.20	27.40	2.00	7.40	24.30	26.00	18.93	14.70	19.23	18.30
Inst‑XL	21.60	34.30	22.40	27.40	18.20	21.20	19.10	27.50	5.00	8.50	15.60	5.90	23.46	16.25	10.00	18.89
GritLM	24.80	32.30	18.90	19.80	17.10	13.60	17.80	29.90	22.00	8.80	25.20	21.20	20.61	25.95	18.40	20.95
Qwen	30.60	36.40	17.80	24.60	13.20	22.20	14.80	25.50	9.90	14.40	27.80	32.90	22.80	17.70	25.03	22.51
Proprietary
Cohere	18.70	28.40	20.40	21.60	16.30	18.30	17.60	26.80	1.90	6.30	15.70	7.20	20.19	14.35	9.73	16.60
OpenAI	23.30	26.70	19.50	27.60	12.80	14.30	20.50	23.60	2.40	8.50	23.50	11.70	20.67	13.00	14.57	17.87
Voyage	23.10	25.40	19.90	24.90	10.80	16.80	15.40	30.60	1.50	7.50	27.40	11.60	19.47	16.05	15.50	17.91
Google	22.70	34.80	19.60	27.80	15.70	20.10	17.10	29.60	3.60	9.30	23.80	15.90	22.54	16.60	16.33	20.00
ReasonIR data
ReasonIR‑8B	26.20	31.40	23.30	30.00	18.00	23.90	20.50	35.00	10.50	14.70	31.90	27.20	24.76	22.75	24.60	24.38
Reason‑ModernColBERT (149 M) reported	33.25	41.02	24.93	30.73	21.12	20.62	20.31	31.07	8.51	9.17	19.51	11.24	27.43	19.79	15.38	22.62
Reason‑ModernColBERT (149 M) our eval**	34.28	41.53	19.96	27.02	21.15	23.62	17.21	26.61	1.32	7.30	19.79	9.70	27.93	13.97	12.26	20.79
SauerkrautLM Reasoning data
SauerkrautLM-Multi-Reason-ModernColBERT (149 M)	36.92	45.53	19.47	27.04	19.35	25.31	20.78	29.74	12.54	10.52	14.62	7.65	28.94	21.14	10.93	22.45
SauerkrautLM‑Reason‑EuroColBERT (210 M)	38.16	39.43	16.99	24.49	17.50	17.60	20.72	29.10	13.57	12.04	10.43	4.95	25.70	21.33	9.14	20.42
SauerkrautLM‑Reason‑Multi‑ColBERT-15m (15 M)	23.33	23.78	10.53	9.03	10.28	10.88	13.13	18.10	15.86	1.75	4.29	0.81	14.64	16.98	2.28	11.81

Remarkable Performance Highlights:

Exceptional in Scientific Domains: Achieves 23.33 (Biology) and 23.78 (Earth Science) - outperforming many models 10-100× its size
Strong StackExchange Performance: With 14.64 Mean StackEx, it matches or exceeds BGE and approaches SBERT-level performance
🐴 PONY CHAMPION: With 15.86, it's the 2nd best model overall on Pony, surpassing ReasonIR-8B (10.50), Qwen-7B (9.90), and only trailing GritLM-7B (22.00) - that's a 15M model outperforming 8 BILLION parameter models!
Efficiency Champion: At just 15M parameters, it delivers 81% of SBERT's performance in scientific domains while being deployable on smartwatches

⚖️ Relative Efficiency

With just 15M parameters, SauerkrautLM-Reason-Multi-ColBERT not only proves that functional retrieval is possible at extreme compression ratios but excels in scientific domains:

Beats E5 (1B) in Biology (23.33 vs 18.60) despite being 67× smaller
Matches proprietary APIs like Cohere in scientific retrieval while being 1000× more efficient
Outperforms BGE across multiple scientific categories despite similar parameter count e baselines while being deployable on edge devices.

BRIGHT Benchmark (German, reasoning‑focused retrieval)

All scores are nDCG@10.

Model / Metric	Biology	Earth	Economics	Psychology	Robotics	Stackoverflow	Sustainable	Leetcode	Pony	AoPS	Theorem‑Q	Theorem‑T	Mean StackEx	Mean coding	Mean theorem	Full Mean
SauerkrautLM‑Multi‑Reason‑ModernColBERT (149 M)	28.00	34.71	12.90	17.98	13.67	19.64	17.70	11.66	15.49	7.27	6.76	1.32	21.15	13.57	5.11	15.59
SauerkrautLM‑Reason‑EuroColBERT (210 M)	31.09	31.48	11.95	18.39	11.25	14.43	20.26	25.67	12.15	9.58	8.15	2.76	19.76	18.91	6.83	16.43
SauerkrautLM‑Reason‑Multi‑ColBERT-15m (15 M)	15.37	20.11	7.36	7.07	4.24	4.71	7.67	0.77	6.31	3.81	0.76	0.00	9.81	3.54	1.52	6.51

Observation: The 15M model, while significantly smaller, maintains functional performance particularly on StackExchange tasks (9.81), demonstrating effective knowledge distillation for specific domains.

NanoBEIR Europe (multilingual retrieval)

Average nDCG@10 across the seven languages we evaluated:

Language	nDCG@10
de	32.98
en	50.94
es	35.30
fr	33.85
it	34.25
nl	32.59
pt	33.76

What makes SauerkrautLM-Multi-ColBERT-15m truly remarkable is not just its tiny size, but its exceptional performance in scientific and technical domains:

Domain-Specific Strengths:

Scientific Retrieval Champion: In Biology and Earth Sciences, it achieves scores (23.33, 23.78) that surpass E5 (1B parameters) and proprietary APIs
Technical Q&A Powerhouse: Mean StackExchange score of 14.64 puts it in the same league as models 10-100× larger
🐴 PONY TASK SUPERSTAR: Ranks #2 globally with 15.86, only behind GritLM-7B (22.00), beating ReasonIR-8B (10.50) and Qwen-7B (9.90)
STEM-Optimized: The knowledge distillation process has successfully preserved critical scientific reasoning capabilities

Bright Performance Comparisons:

Domain	SauerkrautLM-15M	BGE	E5 (1B)	ReasonIR-8B	Qwen-7B	Performance
Biology	23.33	11.70	18.60	26.20	30.60	2× BGE, 125% of E5
Earth Science	23.78	24.60	26.00	31.40	36.40	97% of BGE, 91% of E5
Mean StackEx	14.64	15.17	17.36	24.76	22.80	96% of BGE, 84% of E5
🐴 Pony	15.86	5.70	4.90	10.50	9.90	#2 Overall! 278% of BGE, 151% of 8B model, 160% of 7B!

Why SauerkrautLM-Multi-ColBERT 15m Matters for Edge Deployment

Revolutionary Size: At just 15M parameters, enables late interaction retrieval on smartphones, IoT devices, and embedded systems
Battery-Friendly: Ultra-low power consumption makes it ideal for always-on applications
Privacy-First: Enables on-device search without sending data to the cloud
Rapid Inference: fast query times on modern mobile processors
Multilingual Coverage: Maintains support for 7 European languages despite extreme compression

This model opens entirely new deployment scenarios previously impossible with traditional retrieval models.

Below is a scatter plot that visualises model size (millions of parameters) against BRIGHT Full‑Mean nDCG@10. SauerkrautLM-Multi-ColBERT 15m occupies a unique position as the smallest functional neural retriever on par with SBERT (110M), INST-L (335M) and BGE-bge-large-en-v1.5.

Real-World Impact

The extreme efficiency enables revolutionary applications:

Mobile Search: late interaction retrieval directly on smartphones without cloud connectivity
IoT Integration: Smart home devices with semantic search capabilities
Offline First: Full retrieval functionality without internet access
Privacy Preservation: Sensitive document search stays on-device
Massive Deployment: Deploy on millions of edge devices at negligible cost

📈 Summary: Pushing the Boundaries of Compression

SauerkrautLM-Reason-Multi-ColBERT represents the extreme frontier of model compression in late interaction retrieval. By compressing a ColBERT model to just 15M parameters, we've created a model that:

Achieves functional retrieval performance despite being 500× smaller than SOTA models
Enables entirely new deployment scenarios on edge devices and mobile platforms
Maintains multilingual support across 7 European languages
Delivers fast inference on resource-constrained hardware
Opens late interaction retrieval to billions of devices previously unable to run such models

This model proves that extreme compression is possible while maintaining the core benefits of late interaction retrieval, democratizing access to semantic search technology.

PyLate

This is a PyLate model trained. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.

Indexing documents

First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:

python

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="VAGOsolutions/SauerkrautLM-Reason-Multi-ColBERT",
)

# Step 2: Initialize the Voyager index
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

python

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

python

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

python

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path="VAGOsolutions/SauerkrautLM-Reason-Multi-ColBERT",
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Citation

BibTeX

SauerkrautLM‑Reason‑Multi‑ColBERT-15m

bibtex

@misc{SauerkrautLM-Reason-Multi-ColBERT-15m,
  title={SauerkrautLM-Reason-Multi-ColBERT-15m},
  author={David Golchinfar},
  url={https://huggingface.co/VAGOsolutions/SauerkrautLM-Reason-Multi-ColBERT-15m},
  year={2025}
}

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
  title = {Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
  author = {Reimers, Nils and Gurevych, Iryna},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
  month = {11},
  year = {2019},
  publisher = {Association for Computational Linguistics},
  url = {https://arxiv.org/abs/1908.10084}
}

PyLate

bibtex

@misc{PyLate,
  title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
  author={Chaffin, Antoine and Sourty, Raphaël},
  url={https://github.com/lightonai/pylate},
  year={2024}
}

Acknowledgements

We thank Antoine Chaffin (LightOn AI) for helpful discussions and for clarifying evaluation settings for Reason‑ModernColBERT, and the PyLate team for providing the training framework that made this work possible.