# GPU or Apple Silicon: use BF16
llama-server \
  -hf hotchpotch/bekko-embedding-v1-a8m-GGUF:BF16 \
  --embedding --pooling mean --embd-normalize 2 --ctx-size 8192

bash

curl http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"bekko","input":["What is the tallest mountain in Japan?","Mount Fuji is the tallest mountain in Japan."]}'

For CPU inference, replace :BF16 with :Q8_0. If the selector is unavailable in your llama.cpp version, download the desired GGUF file and pass it with --model:

bash

hf download \
  hotchpotch/bekko-embedding-v1-a8m-GGUF \
  bekko-embedding-v1-a8m-Q8_0.gguf \
  --local-dir .

llama-server \
  --model ./bekko-embedding-v1-a8m-Q8_0.gguf \
  --embedding --pooling mean --embd-normalize 2 --ctx-size 8192

Ollama

The default Ollama tag is BF16 and is recommended for GPU and Apple Silicon users:

bash

ollama pull hotchpotch/bekko-embedding-v1-a8m
curl http://localhost:11434/api/embed \
  -d '{"model":"hotchpotch/bekko-embedding-v1-a8m","input":"What is the tallest mountain in Japan?"}'

For CPU inference, select the Q8_0 tag explicitly:

ollama pull hotchpotch/bekko-embedding-v1-a8m:q8_0

The available tags are latest (BF16), bf16, f16, and q8_0. The registry artifact has been verified as an embedding model with 384 dimensional, L2-normalized output.

Model contract

Architecture: ModernBERT encoder
Context length: 8192 tokens
Embedding dimension: 384
Matryoshka dimensions: 256, 128, 64
Pooling: mean
Similarity: cosine or dot product after L2 normalization
Query/document prefixes: none
Languages: 100+

Use the same unprefixed encoding function for queries and documents.

Quantization quality

Q8_0 was compared with the revision-pinned SentenceTransformers source using the same unprefixed queries and documents:

Benchmark	HF source	Q8_0	Delta
NanoMIRACL-ja NDCG@10	0.667374	0.675283	+0.007908
NanoMIRACL-en NDCG@10	0.641952	0.645472	+0.003520
NanoCoIR representative mean NDCG@10	0.886194	0.886992	+0.000798

The mean cosine similarity to the HF embeddings is 0.999695 over five ordinary multilingual samples. Small positive retrieval deltas should be treated as ranking noise, not as an improvement caused by quantization.

Faster CPU inference

GGUF is intended for portable llama.cpp and Ollama deployment. On a fast x86 CPU, the optimized OpenVINO export is generally faster than llama.cpp GGUF and should be preferred when maximum throughput is the priority. It is available in the hotchpotch/bekko-embedding-v1-a8m source repository when maximum throughput is the priority. The source model also provides the original SentenceTransformers model and its full model card.

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "hotchpotch/bekko-embedding-v1-a8m",
    backend="openvino",
)
embeddings = model.encode(["What is the tallest mountain in Japan?"], normalize_embeddings=True)

Choosing a GGUF format

The repository publishes BF16, F16, and Q8_0. Use BF16 on GPUs and Apple Silicon; use Q8_0 on CPUs, especially CPUs without native BF16 arithmetic. F16 is provided for runtimes or hardware that prefer FP16 explicitly.

Q8_0 remains the best quantized portable trade-off found in our BF16/Q8_0/Q6_K/Q5_K_M/Q4_K_M/IQ4_XS comparison:

Lower-bit K-quants provide almost no size reduction because the 384-wide tensors fall back to Q5_0 or Q8_0. Q4_K_M is only 1.9% smaller than Q8_0.
Lower-bit embeddings have larger cosine drift. IQ4_XS loses 0.0195 mean NDCG@10 on the representative NanoCoIR tasks.
Q8_0 is the fastest quality-preserving quantization on Raspberry Pi 5. Q4_K_M and Q5_K_M reach only 41% and 39% of Q8_0 throughput respectively.
Q6_K has the same size, embeddings, benchmark scores, and effectively the same fallback tensor types as Q8_0, so publishing both would be redundant.

Requested quant	Size MiB	Cosine vs HF	NanoCoIR delta	Pi 5 speed vs Q8_0
Q8_0	113.14	0.999695	+0.000798	1.000x
Q6_K	113.14	0.999695	+0.000798	0.984x
Q5_K_M	111.39	0.997407	+0.002598	0.389x
Q4_K_M	111.03	0.996615	+0.006042	0.408x
IQ4_XS	109.49	0.986278	-0.019549	0.953x

Positive benchmark deltas in this small evaluation are ranking noise rather than evidence that lower numerical precision improves the model.

Files and reproducibility

File	Purpose
`bekko-embedding-v1-a8m-BF16.gguf`	Recommended for GPU and Apple Silicon
`bekko-embedding-v1-a8m-F16.gguf`	FP16 compatibility option
`bekko-embedding-v1-a8m-Q8_0.gguf`	Recommended for CPU inference
`conversion.json`	Source revision, llama.cpp revision, checksums, and config checks

This repository is generated from an immutable source revision. See conversion.json for the exact revision, llama.cpp revision, SHA-256 hash, and release-critical config validation.

For benchmark results, training details, license, and limitations, refer to the source model card.