amaretto-embed-148m — GGUF
GGUF builds of
AmarettoLabs/amaretto-embed-148m
for
llama.cpp — a 148M-parameter EmbeddingGemma specialised for
8 Latin-script languages + code, for running text embeddings on edge / CPU / Jetson hardware.
For the full model description, benchmarks, and the vocabulary-slicing + distillation method, see the
source model card and the
code repository.
Not using llama.cpp? There are also
ONNX builds
for
onnxruntime (fp32 exact, or 297 MB weight-only int8 at ≈0.999), and the
source model itself for PyTorch /
sentence-transformers.
Files
Quantized with llama.cpp. Fidelity is measured against the original PyTorch model — not against the
f16 GGUF, so a subtly wrong conversion couldn't hide behind quants that merely agree with each other.
| file | size | notes |
|---|
amaretto-embed-148m-f16.gguf | 293 MB | exact match to the source model at every length |
amaretto-embed-148m-Q8_0.gguf | 157 MB | near-lossless |
amaretto-embed-148m-Q6_K.gguf | 127 MB | recommended default |
amaretto-embed-148m-Q5_K_M.gguf | 113 MB | smallest; best for short/medium text |
Fidelity is length-dependent, so it is reported per sequence length (cosine vs PyTorch, over a fixed
prompt set spanning nearly the full 2048-token context):
| tokens | f16 | Q8_0 | Q6_K | Q5_K_M |
|---|
| 15 | 1.0000 | 0.9998 | 0.9990 | 0.9969 |
| 135 | 1.0000 | 0.9996 | 0.9982 | 0.9950 |
| 978 | 1.0000 | 0.9993 | 0.9970 | 0.9915 |
| 1822 | 1.0000 | 0.9991 | 0.9960 | 0.9902 |
The f16 build reproduces the PyTorch model exactly at every length, which verifies the conversion
itself; every quant is then measured against that verified anchor.
Choosing a quant. Fidelity drops as sequences get longer, and faster the more aggressive the
quantization. For long documents (≳1000 tokens) prefer Q6_K or higher; Q5_K_M is best suited to
short and medium text. Q8_0 is near-lossless across the whole range.
All builds produce 768-dimensional embeddings (Matryoshka: truncate to 512/256/128 and re-normalize).
The prompt set, reference vectors, and the CI checks that enforce these numbers live in
the code repo.
Usage (llama.cpp)
This is an embedding model with mean pooling and task prefixes — include the prefix that matches
your use (e.g. task: search result | query: for queries, title: none | text: for documents), exactly
as with EmbeddingGemma.
1# one embedding
2llama-embedding -m amaretto-embed-148m-Q6_K.gguf \
3 -p "task: search result | query: how do I sort a list in python?" \
4 --pooling mean --embd-normalize 2
5
6# many at once (one text per line in prompts.txt), JSON output
7llama-embedding -m amaretto-embed-148m-Q6_K.gguf -f prompts.txt \
8 --pooling mean --embd-normalize 2 --embd-output-format json
Also works through the llama-server embeddings endpoint (--embeddings --pooling mean).
Intended use & limitations
A general-purpose text-embedding model — retrieval, semantic search, RAG, clustering, classification, STS —
for English, Spanish, Portuguese, French, German, Italian, Dutch, Polish, and source code.
Not multilingual. The vocabulary for non-target languages was removed; text in other scripts
(Chinese, Japanese, Korean, Arabic, Cyrillic, Greek, Indic, Thai, …) degrades severely. Use the
original google/embeddinggemma-300m for broad multilingual coverage.
Benchmarks (as % of EmbeddingGemma): ≥99.3% on retrieval and STS, ~98.7% on code retrieval, ~97.9% on
classification, ~96% on long-document retrieval. Full tables on the
source model card.
License — Gemma Terms of Use
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms
These GGUF files are a format conversion of
amaretto-embed-148m, itself a Gemma Model Derivative of
google/embeddinggemma-300m. Your use, reproduction,
and distribution are governed by the
Gemma Terms of Use, a copy of
which is distributed here in the
LICENSE file. By using these files you accept those terms.
- Use restrictions (§3.2): you may not use these models in violation of the
Gemma Prohibited Use Policy.
- If you redistribute these or a derivative, the Gemma Terms (§3.1) require you to pass on the use
restrictions, include the
LICENSE, mark modified files, and ship a NOTICE
with the required Gemma notice.
- Trademarks (§4.2): not affiliated with, endorsed by, or sponsored by Google. "Gemma" /
"EmbeddingGemma" are used descriptively to identify the upstream model.
Built by
AmarettoLabs. Converted with
llama.cpp.
