A tiny embedding model that punches far above its weight.
Named after the colibri (hummingbird) — the smallest bird, yet it out-flies much larger ones.
~157M params · 768-dim · plain SentenceTransformer (no adapters) · runs on a cheap CPU VPS
Colibri is a compact Brazilian-Portuguese text-embedding model derived from
google/embeddinggemma-300m. Despite being half the size,
it matches or beats much larger multilingual embedders on MTEB-BR — including 7B and 27B models — and is
designed small on purpose so it runs comfortably on a simple CPU VPS, with no GPU required.
Use cases: Portuguese semantic search, retrieval-augmented generation (RAG), retrieval, reranking,
clustering, semantic textual similarity (STS) and classification — for Brazilian-Portuguese text across legal,
tax, financial, medical, academic and general domains.
✨ Highlights
🏆 Beats bigger models on MTEB-BR — including embeddinggemma-300m, and 7B / 27B multilingual embedders.
🪶 Half the footprint of embeddinggemma-300m: ~607 MB vs ~1.2 GB, and less RAM.
💻 Runs on a $-few/month CPU VPS — benchmarked on 4 vCPU / 16 GB (no GPU).
🔌 Plain SentenceTransformer — no adapters, no LoRA at inference. Drop-in.
📐 Matryoshka dimensions (768 / 512 / 256 / 128) + an fp16 branch + a fast ONNX path.
🥇 Quality — MTEB-BR
Evaluated on MTEB-BR (22 native Brazilian-Portuguese tasks:
retrieval, reranking, STS, classification, clustering, pair-classification). Score = mean over the 22 tasks.
Model
Params
MTEB-BR
🐦 Colibri (this model)
~157M
0.6501
google/embeddinggemma-300m
300M
0.6490
Linq-AI-Research/Linq-Embed-Mistral
7B
0.6473
openai/text-embedding-3-large
–
0.6449
intfloat/multilingual-e5-large-instruct
560M
0.6409
Salesforce/SFR-Embedding-2_R
7B
0.6397
Alibaba-NLP/gte-Qwen2-7B-instruct
7B
0.6392
microsoft/harrier-oss-v1-27b
27B
0.6390
BAAI/bge-m3
568M
0.6157
intfloat/multilingual-e5-large
560M
0.5909
A ~157M-parameter model outperforming 7B–27B multilingual embedders on Brazilian Portuguese.
Size ↔ quality frontier. Colibri sits on the open-model Pareto frontier for MTEB-BR — it dominates its own
baseembeddinggemma-300m (half the size, higher score) and matches models up to ~10× larger:
💻 Built for a simple CPU VPS
Colibri was designed to be small enough for cheap, GPU-less hosting. Benchmarked head-to-head against
google/embeddinggemma-300m on a VerdaCPU.4V.16G (4 vCPU, 16 GB RAM) —
a typical small VPS — encoding Brazilian-Portuguese sentences:
Precision
Model
Model size
Peak RAM
Latency (p50)
Throughput
fp32
🐦 Colibri
607 MB
969 MB
76 ms
37 sent/s
fp32
embeddinggemma-300m
1211 MB
1253 MB
78 ms
36 sent/s
ONNX
🐦 Colibri
~600 MB
2.9 GB
33 ms
49 sent/s
ONNX
embeddinggemma-300m
~1200 MB
5.3 GB
38 ms
41 sent/s
Takeaways
Half the size and less RAM than embeddinggemma-300m — fits a smaller, cheaper VPS.
Same encode latency at equal precision (the vocabulary trim shrinks the token-embedding matrix, not the
transformer compute) — you lose nothing in speed.
The ONNX backend roughly doubles throughput (33 ms vs 76 ms) — best when latency matters; fp32 is the
most RAM-frugal.
Benchmark hardware: VerdaCPU.4V.16G. Numbers are indicative
single-node CPU measurements.
🚀 Usage
python
1from sentence_transformers import SentenceTransformer
23model = SentenceTransformer("tardellirs/colibri-embed-ptbr")45query ="Como declarar imposto de renda de aluguel?"6docs =[7"Rendimentos de aluguel devem ser informados na ficha de Rendimentos Tributáveis...",8"O Pix é um meio de pagamento instantâneo criado pelo Banco Central...",9]1011q = model.encode(query, prompt_name="query")# "task: search result | query: "12d = model.encode(docs, prompt_name="document")# "title: none | text: "13print(model.similarity(q, d))
Faster on CPU — ONNX backend (≈2× throughput, no code changes to your pipeline):
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", backend="onnx")
Matryoshka dimensions — truncate the 768-dim output to 512 / 256 / 128 for smaller vectors (cheaper index,
faster search) with a graceful quality trade-off:
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", truncate_dim=256)
(The MTEB-BR score above is for the full 768-dim embeddings.)
🔧 How it was built
A three-stage pipeline: vocabulary trimming → knowledge distillation → model soup.
2. Knowledge distillation. Relational (similarity-preserving) KD from two complementary teachers —
Qwen/Qwen3-Embedding-4B (strong at clustering) and
Qwen/Qwen3-Embedding-8B (strong at retrieval / reranking). The
student learns to reproduce the average of the two teachers' pairwise-similarity matrices (dimension-agnostic;
preserves STS). Trained on a diverse ~100k-passage corpus of native Brazilian Portuguese spanning legal, tax,
financial, medical, academic, technical, encyclopedic and general-web domains:
Evaluation integrity: the model is trained only on training / non-evaluation splits — every MTEB-BR
evaluation example is held out, so the scores reflect generalization, not memorization.
3. Model soup + merge. Distillation checkpoints were combined (model soup / weight averaging) and linearly
merged with the trimmed base, with the mixing weight chosen on held-out validation.
Both operations yield ordinary weights, so the published model is a single standalone encoder.
