m2v-bge-m3-edu — static embeddings for educational content
A fast, torch-free static embedding model tuned for educational metadata. It is a
Model2Vec distillation of
BAAI/bge-m3 enriched with a domain vocabulary of
curated education-domain keyword tags, so that subject terms and multi-word phrases
(e.g. "Quadratische Gleichung", "Erneuerbare Energien") are represented as atomic units.
The base model (bge-m3) is multilingual, so the model still embeds many languages; the
added domain vocabulary, however, is primarily German, so the strongest gains are on
German educational text. Inference is numpy-only — no PyTorch required — and much
faster and lighter than a transformer.
How it was built
Base model:BAAI/bge-m3 (the Model2Vec team's
recommended base for multilingual data).
Distillation:model2vec.distill with
CLS / first pooling (matching bge-m3's native pooling — this choice has a large
effect on quality), pca_dims=512, float32, SIF weighting (sif_coefficient=1e-4).
Domain vocabulary: ~70,000 cleaned keyword tags (document frequency ≥ 3) extracted
from the keyword field of ~340k educational resources — mostly German subject terms —
added on top of bge-m3's subword vocabulary. Subword fallback is kept for out-of-vocabulary text.
Evaluation
Evaluated on an educational subject-classification task and compared against other
embedding models.
Methodology (identical across models — only the embedding model differs): 25,068
educational items with 47 subject labels and a fixed train / validation / held-out test
split. Each embedding model is frozen; a small MLP classification head is trained on top
(Optuna hyperparameter search on validation), per-label decision thresholds are tuned on
validation, and F1 is reported on the held-out test split that is never used for
training or tuning (honest, non-optimistic evaluation).
embedding model
dim
F1-macro
F1-micro
inference
intfloat/multilingual-e5-small
384
0.587
0.777
requires torch
m2v-bge-m3-edu (this model)
512
0.581
0.746
torch-free (numpy)
m2v-gte-multilingual-768 (generic static)
768
0.559
0.747
torch-free (numpy)
What this shows: the education-domain vocabulary lifts macro-F1 over a generic
multilingual static baseline (0.559 → 0.581). A transformer (e5-small) remains slightly
ahead on this task — an expected trade-off for static embeddings — but this model needs
no PyTorch and is far lighter at inference. In our CPU benchmark it encoded ~400 texts/s
single-threaded (BLAS capped to one core); throughput scales with cores.
For this specific classification task, a plain TF-IDF + linear classifier scores higher
still (macro ≈ 0.62). So the intended value of this model is fast, torch-free semantic
embeddings for the education domain, not being the single best classifier for one dataset.
These are domain-specific benchmark numbers, not a general-purpose (e.g. MTEB) score.
Intended use
Fast embedding of (mostly German) educational metadata — titles, descriptions, keywords —
for classification, clustering, semantic search and retrieval, especially in CPU-only or
low-resource deployments where a transformer is too heavy.
Model2Vec passes a vocabulary through a
sentence-transformer, reduces dimensionality with PCA, and weights the token embeddings
with SIF weighting. At inference it takes the
(weighted) mean of the static token vectors of a text — no transformer forward pass.
License
Inherits the base model's license (bge-m3: MIT).
Citation
@article{minishlab2024model2vec,
author = {Tulkens, Stephan and {van Dongen}, Thomas},
title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
year = {2024},
url = {https://github.com/MinishLab/model2vec}
}
