You can deploy this model and start earning money today!

cstr/
lfm2-embed-GGUF

Name: lfm2-embed-GGUF
Availability: InStock

gguf

embedding

retrieval

text-embeddings-inference

crispembed

LiquidAI/LFM2.5-Embedding-350M

quantized

other

Views

No views yet

LFM2.5-Embedding-350M — CrispEmbed GGUF

CrispEmbed-native GGUF quantizations of LiquidAI/LFM2.5-Embedding-350M.

Format note: These GGUFs use CrispEmbed's internal tensor naming (lfm.* prefix, arch=lfm2). They are not interchangeable with the official LiquidAI GGUFs which target llama.cpp (lfm2-bidir arch, blk.* tensor naming). Use the LiquidAI GGUFs if you want llama.cpp/llama-server.

Files

File	Size	Description
`lfm2-embed-q8_0.gguf`	359 MB	8-bit quantization — best accuracy, recommended
`lfm2-embed-q4_k.gguf`	222 MB	4-bit K-quant — 3× compression, minimal quality loss
`lfm2-embed-f16.gguf`	678 MB	Full fp16 — reference precision

Parity (CrispEmbed q8_0 vs HF float32 `Lfm2BidirectionalModel`)

Stage	Cosine	Notes
per-layer (all 20)	≥ 0.9999	measured on 3-token input via test-lfm2-diff
CLS embedding q8_0	0.9999	5 diverse test sentences
CLS embedding q4_k	0.982	expected q4_k quantization floor

Model

Architecture: 16-layer hybrid (10 ShortConv + 6 GQA attention), hidden=1024
Pooling: CLS token (position 0) of last hidden state, L2-normalized
Dimension: 1024
Languages: 11 (en, de, fr, es, it, pt, nl, pl, ru, ja, zh)
Parameters: 350M
Task prefixes: "query: " for queries, "document: " for passages

Usage with CrispEmbed

CLI

bash

# Download
./crispembed --download lfm2-embed

# Embed a query (prefix auto-applied)
./crispembed -m ~/.cache/crispembed/lfm2-embed-q8_0.gguf "What is the capital of France?"

# Embed a document (disable auto-prefix and supply explicitly, or use --prefix)
./crispembed -m ~/.cache/crispembed/lfm2-embed-q8_0.gguf \
  --prefix "document: " "Paris is the capital of France."

# JSON output for downstream use
./crispembed -m ~/.cache/crispembed/lfm2-embed-q8_0.gguf --json "query: machine learning"

Python (via crispembed Python bindings)

python

import crispembed

model = crispembed.load("~/.cache/crispembed/lfm2-embed-q8_0.gguf")

query_emb = model.encode("query: What is the capital of France?")
doc_emb   = model.encode("document: Paris is the capital of France.")

import numpy as np
score = np.dot(query_emb, doc_emb)  # both are already L2-normalized
print(f"Similarity: {score:.4f}")

Rust

rust

use crispembed::CrispEmbed;

let model = CrispEmbed::load("lfm2-embed-q8_0.gguf")?;
let emb = model.encode("query: hello world")?;

Comparison with official LiquidAI GGUFs

	This repo	LiquidAI/LFM2.5-Embedding-350M-GGUF
Runtime	CrispEmbed	llama.cpp / llama-server
GGUF arch tag	`lfm2`	`lfm2-bidir`
Tensor naming	`lfm.*` prefix	`blk.*` / llama.cpp convention
Quantizations	f16, q8_0, q4_k	BF16, F16, Q4_0, Q4_K_M, Q5_K_M, Q6_K, Q8_0
q8_0 size	359 MB	379 MB
Metal GPU	Yes (Apple Silicon)	Yes

Conversion

Convert from the source model yourself: