paraphrase-multilingual-MiniLM-L12-v2 — ExecuTorch
Paraphrase similarity across 50+ languages, MiniLM-sized. Text in, one
384-dimensional vector out, for search and retrieval that never leaves the
device.
- Source: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 — 12 layers, 384 dimensions, 250,037 vocabulary
- License: apache-2.0
- Input:
input_ids and attention_mask, both [1, 256] int64
- Output:
[1, 384], mean-pooled and not normalised inside the graph
The recipe is in the graph, and it was read off this repo
sentence-transformers stores it per model, and the shelf's seven embedding models do
not agree. This one pools mean and
does not normalise, read from
1_Pooling/config.json and modules.json rather than inferred from the family name.
Getting it wrong does not throw; it returns vectors that look fine and rank wrong.
Verification
| build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
|---|
| fp32 | embed_paraphrase_multilingual_xnnpack_fp32.pte | 470.2 | 21.2 | 79.0% | 1.000000 | 0% |
| fp16 | embed_paraphrase_multilingual_xnnpack_fp16.pte | 235.2 | 31.2 | 68.2% | 1.000000 | 22% |
*Mac arm64, one 256-token sequence, fastest of five medians of ten — a reference
point for relative cost, not a device number. The host shares its cores with other work,
and a single median does not survive that: the same eager model here measured 19.6 ms and
182.8 ms twenty minutes apart. Contention only ever adds time, so the fastest repetition is
the one that means something. Torch eager fp32, measured the same way, is
18.8 ms.
Cosine is measured against the model run in eager through its own pooling, over eight
sentences. The last column is the one that decides: rank those eight against each
other, and ask whether this build's score error is smaller than the gap between the
document a query retrieves and the runner-up. Every shipped build keeps all eight
top-1 results.
The attention is eager, and that is the faster export
F.scaled_dot_product_attention does not survive export as one operation. The edge
dialect lowers it through _safe_softmax, whose guard against a row with no unmasked key
at all leaves 11 operations XNNPACK cannot take, in every attention
block — scalar_tensor, where, mul.Scalar, logical_not, eq, full_like, any.dim. Each one cuts the subgraph in two.
The switch is attn_implementation="eager": transformers then builds the mask
itself, as torch.finfo(dtype).min, instead of handing F.sdpa a boolean mask
for PyTorch to fill with -inf.
The guard is emitted whether or not it can ever fire, and here it cannot: it triggers only
on -inf, and this arm never produces one. So the two differ only about rows that have no
unmasked key at all — sdpa zeroes them, this one gives them a uniform row — and those are
padding rows, which the pooling discards and which every real query row masks out anyway.
Measured with all but eight positions masked, as adversarial as this shape gets, the two
graphs agree to 1.4e-07.
XNNPACK fp32 goes from 62.9% to 79.0% delegated.
Not shipped: int8
embed_paraphrase_multilingual_xnnpack_int8.pte is 406.7 MB against fp16's 235.2 MB. Dynamic int8 quantises the
linear weights and leaves the token embedding table in fp32, and here that table is
384 MB of the 470.2 MB model — 82%. The size a build comes out at is
0.5 + 1.5 x (table share) times the fp16 build; at 82% that is
1.73, so there was never a smaller file to be had.
It is withheld on the number that decides. Ranking the eight test sentences against
each other, this build moves a pair score by at most 0.0061 while the
closest fp32 decision — the gap between the document a query retrieves and the
runner-up — is 0.0009. That is 648% of
the room available, against a bar of 50%.
Correlation reads 0.999744 for this build, which no correlation gate
would stop.
Not shipped: Core ML (fp16, iOS)
embed_paraphrase_multilingual_coreml_all.pte converts, runs and agrees with eager to a cosine of
0.999994 — and is withheld anyway, on the number that decides. Ranking
the eight test sentences against each other, it moves a pair score by at most
0.0008, while the closest fp32 decision — the gap between the document a
query retrieves and the runner-up — is only 0.0009. That is
87% of the room available, against a bar of 50%.
The shift is tiny in absolute terms; this model's decisions are tinier. That is a
property of this model on this set rather than of the build — two of the eight
sentences are near-ties for it, and no other model on this shelf comes within a
factor of two of a decision that narrow.
torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
(conversion scripts:
executorch-models)
