Views
No views yet
open-ogma family is the permissive successor to the axiotic/ogma-* family
(which is CC-BY-NC-4.0 due to its training-data mix).[QRY]/[DOC]/[SYM]) + two projection heads:
proj_small 128→384 and proj_large 128→1024.proj_small (384d) — the head used for the headline
MTEB row. High-fidelity mode: proj_large (1024d) scores +0.010 mean /
+0.023 retrieval over the default — opt in when output width is not a
constraint.| output | dim | mean | Class | Clust | Pair | Rerank | Retr | STS | Summ |
|---|---|---|---|---|---|---|---|---|---|
proj_large (opt-in) | 1024 | 0.5202 | — | — | — | — | 0.2823 | — | — |
proj_small (default) | 384 | 0.5104 | — | — | — | — | 0.2589 | — | — |
| trunk (pre-head) | 128 | 0.5057 | — | — | — | — | 0.2566 | — | — |
minishlab/potion-base-8M
at 0.5328 vs its official 53.33): teacher bge-small-en-v1.5 (33M) scores
0.6368 and bge-large-en-v1.5 (335M) 0.6528 — this model retains ~82% of
the small teacher's mean at 7.5% of its parameters.BAAI/bge-small-en-v1.5 (384d) and BAAI/bge-large-en-v1.5
(1024d) — each supervised through its own projection head with equal loss
weight (0.5/0.5), on a 10.6M-document permissive blend of C4 and FineWeb-Edu
(contamination-flagged against MTEB test sets, 0.11% hits marked). Loss:
matryoshka-weighted distillation (α) + in-batch contrastive (β) on a
1.0→0.7/0.3 schedule; task-token mix {QRY 0.25, DOC 0.25, SYM 0.5}; AdamW
lr 5e-4 cosine, wd 0.01.| component | license |
|---|---|
teacher BAAI/bge-small-en-v1.5 | MIT |
teacher BAAI/bge-large-en-v1.5 | MIT |
| corpus C4 (allenai/c4) | ODC-BY |
| corpus FineWeb-Edu (HuggingFaceFW) | ODC-BY |
| tokenizer (ogma sentencepiece, trained in-project) | MIT |
| these weights | MIT |
1# clone this repo, then:
2from ogma_libre import OgmaLibre
3model = OgmaLibre.from_repo("path/to/open-ogma-micro")
4
5embs = model.encode(["A quick brown fox."]) # (N, 384) default
6embs = model.encode(["A quick brown fox."], head="large1024") # (N, 1024) best quality
7embs = model.encode(["A quick brown fox."], head="base") # (N, 128) trunk
8
9# Retrieval convention: encode queries with task="qry", documents with task="doc".
10q = model.encode(["what is a fox?"], task="qry")
11d = model.encode(["The fox is a small canid."], task="doc")model.safetensors (weights), config.json (architecture + heads),
tokenizer/ogma_sp.model (sentencepiece), ogma/ (vendored model code),
ogma_libre.py (loader).axiotic/open-ogma-small — the 8.96M-parameter sibling (same recipe, small trunk): 0.5843 @1024d.