You can deploy this model and start earning money today!

smarttasks/
all-MiniLM-L6-v2-GGUF

Name: all-MiniLM-L6-v2-GGUF
Availability: InStock

sentence-transformers/all-MiniLM-L6-v2

quantized

endpoints_compatible

Views

No views yet

all-MiniLM-L6-v2 — Embedding GGUF (quantization-verified)

Quantized embedding model in GGUF, served in --embedding mode via llama.cpp. This is an encoder — it outputs vectors, not text. It is validated for retrieval quality and quantization fidelity, not chat behavior.

Files

all-MiniLM-L6-v2-Q4_K_M.gguf (21.3 MB)
all-MiniLM-L6-v2-Q5_K_M.gguf (22.0 MB)
all-MiniLM-L6-v2-Q8_0.gguf (25.3 MB)

Quantization drift (vs f16)

Mean cosine similarity of embeddings vs the f16 baseline. 1.0 = identical.

Quant	Mean cosine	Min cosine	Verdict
Q4_K_M	0.99303	0.99179	excellent (>0.99)
Q5_K_M	0.99421	0.99224	excellent (>0.99)
Q8_0	0.99965	0.99956	excellent (>0.99)

Retrieval sanity (lightweight)

Built-in 12-query retrieval check (no external corpus): top-1 accuracy 1.0, MRR 1.0. healthy (top-1 >= 0.9)

Retrieval (MTEB)

Standardized MTEB retrieval scores (main metric, usually nDCG@10 — higher is better). These are comparable across models on the MTEB leaderboard.

Task	Score
SciFact	0.6507

Metric: main_score (retrieval tasks: nDCG@10). Measured on the Q8_0 quant served via llama.cpp.

What this is NOT

This card carries no safety, red-team, or viewpoint scores: those do not apply to an embedding model. For chat-model governance cards, see the SmartTasks text-LLM line.

smarttasks/all-MiniLM-L6-v2-GGUF