Zero-shot Named Entity Recognition — detect arbitrary entity types at inference time, no retraining needed.
All variants produce the same entities. Score deltas vs F32: Q8_0 ≤ 0.01, Q4_K ≤ 0.03.
LFM2.5-350M bidirectional backbone (16 layers: 10 ShortConv + 6 GQA attention, SwiGLU FFN) + layer fusion (squeeze-and-excitation) + BiLSTM + GLiNER span-label matching head.
1# CLI
2./crispembed -m gliner-lfm-q8_0.gguf \
3 --ner "Tim Cook announced the new iPhone in Cupertino" \
4 --ner-labels "person,organization,location,product" --json
5
6# Server
7./crispembed-server --ner gliner-lfm-q8_0.gguf --port 8080
8curl -X POST http://localhost:8080/ner/extract \
9 -d '{"text": "Tim Cook at Apple", "labels": ["person", "organization"]}'
1from crispembed import CrispNER
2
3ner = CrispNER("gliner-lfm-q8_0.gguf")
4entities = ner.extract(
5 "Maria Schmidt arbeitet bei Siemens in München",
6 labels=["person", "organization", "location"],
7)
8for e in entities:
9 print(f"{e['text']} => {e['label']} ({e['score']:.2f})")
All 16 backbone layers cos=1.000000 vs HuggingFace Python reference. 17/17 entities match across 5 test texts.
LFM Open License v1.0 — free for entities under $10M annual revenue. See
upstream license.
1python models/convert-gliner-lfm-to-gguf.py \
2 --model VAGOsolutions/SauerkrautLM-LFM2.5-GLiNER \
3 --output gliner-lfm-f32.gguf
4./crispembed-quantize gliner-lfm-f32.gguf gliner-lfm-q8_0.gguf q8_0
