The embedding model used by FlowGraph — a
local-first desktop app that gives AI tools persistent, graph-shaped memory stored on the user's own
machine.
These are FlowGraph's own trained weights: a MiniLM-L12 student distilled from a
permissively-licensed offline teacher. No third-party model is redistributed here. The model runs
locally and in-process via ONNX Runtime — no API key, no network call at inference time.
What's in this repo
A single packaged bundle, embedder-onnx-bundle-a2-student-l12.tar.gz (~124 MB), which FlowGraph's
installer downloads by pinned digest. Inside, at the archive root:
MiniLM-L12 (12 layers, hidden 384, 12 heads, ~34M params) → mean pooling → Dense(384 → 768, no bias, identity activation) → L2 normalize. The exported ONNX bakes the whole pipeline in: it takes
input_ids + attention_mask and emits a single L2-normalized 768-d sentence_embedding per row.
There is no token_type_ids input.
dimensions: 768
max sequence length: 256
text prefix: search_document: — prepended to both passages and queries (see below)
1import numpy as np, onnxruntime as ort
2from tokenizers import Tokenizer
34tok = Tokenizer.from_file("tokenizer.json"); tok.enable_truncation(max_length=256)5sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])67defembed(texts):8 enc =[tok.encode("search_document: "+ t).ids for t in texts]# NOTE: same prefix for queries9 L =max(len(e)for e in enc)10 ids = np.zeros((len(enc), L), dtype=np.int64)11 mask = np.zeros((len(enc), L), dtype=np.int64)12for i, e inenumerate(enc):13 ids[i,:len(e)]= e; mask[i,:len(e)]=114return sess.run(None,{"input_ids": ids,"attention_mask": mask})[0]# already unit-normalized
The prefix is symmetric. Unlike nomic-class models, this student was distilled on a single
search_document: regime, so queries take the same prefix as passages. Using search_query: puts
the input out of distribution.
Training
Distilled from nomic-ai/nomic-embed-text-v1.5 at revision
e9b6763023c676ca8431644204f50c2b100d9aab (Apache-2.0), used strictly as an offline teacher — its
weights are not redistributed.
loss
pointwise MSE to the teacher's L2-normalized embedding + relational in-batch pairwise-similarity MSE (match the teacher's rankings, not just its coordinates)
corpus
wikimedia/wikipedia, config 20231101.simple, split train; texts with len > 100, truncated to the first 512 chars, first 150,000 in dataset order
epochs / batch / lr / warmup
4 / 128 / 2e-4 / 100
precision
AMP (fp16), single GPU
max seq length
256
student init
sentence-transformers/all-MiniLM-L12-v2 + the Dense projection head + Normalize
The relational term is what made this work: plain MSE-to-embedding plateaued around 59% on an internal
doc-level proxy regardless of student size (23M / 34M / 110M all landed in the same band), while adding
the in-batch pairwise-similarity term lifted the 34M student to ~71% — about 95% of the teacher at
roughly a quarter of the parameters.
Reproducibility. No RNG seed was set during training, so bit-identical retraining is not possible
by construction. The contract is: the released checkpoint is pinned by hash (above) and the
procedure is documented; reproduction means equivalent under this procedure, not bit-identical.
Export
torch.onnx.export (legacy exporter, dynamo=False), opset 17, do_constant_folding=True, dynamic
axes {0: batch, 1: seq} on the inputs. Attention is forced to eager at export time. (For the
record: an earlier SDPA-traced export produces numerically identical vectors — cosine 1.00000 and 100%
top-10 neighbour overlap across a probe set — so the attention implementation is pinned for procedure
fidelity, not because it changes the vector space.)
Faithfulness was verified against the PyTorch reference on 200 real corpus texts (including 73 short,
label-like strings): min cosine 1.00000, mean 1.00000, top-10 neighbour overlap 100.0%.
Evaluation
Measured inside FlowGraph's retrieval benchmark at node level, with the graph held constant and only
the embedder swapped (so the numbers isolate the embedder rather than the ingest), 200 labeled queries
per dataset, k=10:
ArguAna (1000 distractors)
HotpotQA
this model — vector recall@10
38.5%
60.5%
FlowGraph's previous static embedder
30.5%
31.0%
teacher (nomic-embed-text-v1.5)
48.5%
65.3%
this model — hybrid nDCG@10
0.330
0.611
teacher — hybrid nDCG@10
0.340
0.612
On raw vector recall the student reaches 79–93% of its teacher. On the fused hybrid channel the app
actually serves (reciprocal-rank fusion of vector + BM25 + graph walk) it reaches 97–100% of the
teacher, and slightly exceeds it on HotpotQA MRR. Note that these are node-level scores over an
LLM-extracted knowledge graph, not standard BEIR document retrieval numbers, so they are not comparable
to published BEIR leaderboards.
Inference is CPU-cheap: roughly 25 ms/text single-threaded on a 2-core container for short inputs.
Intended use & limitations
Built for retrieval over a personal knowledge graph: short node labels and document chunks, English,
≤256 tokens. It is a distilled student — on raw semantic recall the teacher is still better, and it
inherits the teacher's and the Simple-English-Wikipedia corpus's biases. Not evaluated for
classification, clustering, or non-English text.
License
Apache-2.0. The weights are FlowGraph's own; the teacher was used offline under its own Apache-2.0
terms and is not redistributed here.
