urchade/gliner_large-v2.1 fine-tuned to redact PII and secrets in Privileged Access Management support text: Jira tickets, log excerpts, config snippets, CLI transcripts, audit events. Turkish-weighted, bilingual (TR/EN).

Span-mode architecture, so unlike DeBERTa token-level GLiNER variants this one exports to ONNX with bit-level fidelity.

Results

3,000 held-out cases / 7,105 gold spans, threshold 0.4, type-aware overlap matching:

	base	fine-tuned
micro F1	0.365	0.907
micro precision	0.490	0.874
micro recall	0.291	0.942
macro F1	0.347	0.892
detection recall (type-agnostic)	0.500	0.994

Of 50 labels: 16 reach F1 >= 0.95, 30 reach >= 0.90, only 3 fall below 0.70.

Strongest	F1
`IPV6`	1.000
`VAULT_PATH`	0.995
`INTERNAL_IP`	0.994
`CONNECTION_STRING`	0.990
`CLOUD_RESOURCE`	0.990
`LDAP_DN`	0.978
`HOSTNAME`	0.978
`WEBHOOK_URL`	0.977
`PUBLIC_IP`	0.977
`JIRA_USER`	0.974

Weakest	F1
`VKN`	0.795
`DOB`	0.782
`ENCRYPTION_KEY`	0.776
`DOMAIN`	0.774
`TCKN`	0.757
`SNMP_COMMUNITY`	0.690
`SGK_NO`	0.673
`ACCOUNT_NUMBER`	0.626

Numeric-identifier types are the weak spot (ACCOUNT_NUMBER, SGK_NO, TCKN) — an English-vocabulary tokenizer fragments Turkish digit-strings poorly.

Fastest inference — read this first

Everything below is measured on this checkpoint, not estimated. Hardware: RTX 4090 / Intel Xeon Gold 6330 (AVX512-VNNI, 16 threads) / Apple M4 Pro.

1. The single biggest lever is the number of labels you pass

GLiNER prepends every label description to the input sequence, so cost scales with label count, not just text length. This is free accuracy-wise:

Labels passed	CPU latency	Speedup
50 (all)	762 ms	—
10	385 ms	2.0x

Filter to the tiers you actually need. label_map.json in this repo groups the 50 labels; a two-pass design (secrets first, identity/infra only if needed) keeps you near 385 ms with no loss.

2. Device choice

Setup	Latency/case
RTX 4090, fp32	23 ms
Apple M4 Pro, MPS, 10 labels	158 ms
Xeon 6330 CPU, 16 threads, 10 labels	385 ms
Xeon 6330 CPU, 16 threads, 50 labels	762 ms

On CPU, 8-16 threads is the sweet spot — more threads made it slower on both an M4 Pro (14 threads worse than 8) and a 48-core EPYC. Apple M4 Pro CPU beat a server EPYC 7K62 by 35%; this workload is bound by memory bandwidth and single-core speed, not core count.

3. ONNX: use it for portability, not speed

onnx/model.onnx (fp32) is numerically exact — verified end-to-end through the full decode pipeline: identical F1, precision, recall, and even identical prediction count (897 of 897) versus PyTorch. Tensor-level max|Δ| = 0.0004.

But it is 5.2x slower than PyTorch on CPU (4070 ms vs 762 ms at 50 labels). Root cause confirmed from onnxruntime source: its transformer optimizer's MODEL_TYPES list has no deberta entry, so the graph gets zero attention fusion and DeBERTa's relative-position machinery executes as a long unfused Gather/Einsum/Where chain.

Export it if you need C#/Java/Go or an ORT serving stack. Do not export it for speed.

Six ONNX inputs: input_ids, attention_mask, words_mask, text_lengths, span_idx, span_mask -> logits.

4. Do not quantize this model. No int8 files are shipped.

Seven configurations were measured. All seven destroy the model. The failure is always the same: logit scale collapses so the maximum logit goes negative, and nothing passes a sigmoid threshold — the model silently returns empty results rather than erroring.

Configuration	Result
torch dynamic int8, all `nn.Linear`	F1 0.920 -> 0.002
torch dynamic int8, FFN only (attention excluded)	F1 -> 0.000
torch dynamic int8, FFN + heads	F1 -> 0.000
ONNX dynamic QInt8	897 predictions -> 9
ONNX static QDQ uint8, min/max calibration	logit max +46.4 -> -0.514, positives 6 -> 0
ONNX static, percentile calibration	calibrator crashes on this graph
ONNX static, MatMul-only	logit max -> -1.121, positives 6 -> 0

Lowering the threshold does not rescue it: at 0.05 you get scattered predictions with F1 0.010-0.014.

Why, and when int8 does work

This is not an intrinsic DeBERTa property. The discriminating evidence is GLiNER issue #270: with the same script, gliner_small-v2.1 (deberta-v3-small) quantizes cleanly (0.98 -> 0.98), while gliner_multi-v2.1 (mdeberta-v3-base) collapses (0.99 -> 0.21). Same disentangled attention, different scale.

The mechanism is activation outliers in the post-LayerNorm residual stream, whose magnitude grows with model scale. Per-tensor activation quantization lets one outlier channel set the scale, leaving everything else 2-3 usable levels. Per-channel weight quantization cannot fix it because the damage is activation-side.

Note also that issue #218 reports this exact symptom and is open with no maintainer response, and that knowledgator/gliner-pii-large-v1.0's own published onnx/model_quint8.onnx shows max logit -1.56 — i.e. published quantized GLiNER artifacts are broken too. Shipping a quantized file is not evidence it works.

If you need int8, start from a -small backbone, which is the only size with published evidence of surviving quantization.

5. What we measured and rejected

Approach	Measured outcome
`modern-gliner-bi-large-v1.0` (ModernBERT bi-encoder)	19x slower on CPU (14430 ms vs 762 ms)
`modern-gliner-bi-base-v1.0`	3x slower (2281 ms), half the parameters
FP8 on RTX 4090 (Ada sm_89)	tooling is Hopper/decoder-centric; torchao float8 measured zero speedup on sm_89

ModernBERT's advantage comes from GPU unpadding + FlashAttention, neither of which applies on CPU; and its bi-encoder re-encodes labels per call in the predict_entities path, so you pay for two encoders instead of one. Its tokenizer is also worse for Turkish (18 tokens vs 14 for DeBERTa on the same sentence).

Untested but plausible for CPU: Intel Neural Compressor's accuracy-aware tuning, which took deberta-v3-base to int8 at +0.17% accuracy and 1.26x throughput on MRPC — but note MRPC is argmax-based and therefore insensitive to the exact logit-scale collapse that breaks this model.

Usage

python

from gliner import GLiNER
model = GLiNER.from_pretrained("omeryentur/gliner-pam-pii-large")

labels = ["password", "login account name or username",
          "private internal ip address", "database connection string with credentials",
          "password vault safe or object path"]

text = ("Environment : 10.20.42.121\nuser : admin@host\npasswd : kr10ipsla\n\n"
        "pamuser / kr10ipsla")
for e in model.predict_entities(text, labels, threshold=0.4):
    print(e["label"], "->", e["text"], round(e["score"], 2))

Labels are natural-language descriptions (GLiNER is open-vocabulary). All 50 strings used in training are in label_map.json — use them verbatim for best results. Threshold 0.4 is balanced; 0.3 raises recall for masking scenarios.

The `max_width` fix — essential if you retrain

Span-mode GLiNER only predicts spans up to max_width tokens. The base config is 12, which makes 12.6% of this dataset's entities structurally unlearnable — concentrated in exactly the PAM-critical labels (LDAP_DN 100% over, CONNECTION_STRING 98%, WEBHOOK_URL 88%, VAULT_PATH 57%).

SpanMarkerV0's weights are width-independent (no width embedding table), so max_width can be raised without breaking the pretrained checkpoint. This model uses max_width=48 (99.87% entity coverage). The result:

Label	base (mw=12)	this model (mw=48)
`CONNECTION_STRING`	0.000	0.990
`CLOUD_RESOURCE`	0.204	0.990
`LDAP_DN`	0.083	0.978
`VAULT_PATH`	0.151	0.995
`IPV6`	0.420	1.000

Cost: span candidates scale with seq_len x max_width, so batch size must drop (bs 16 with gradient checkpointing fits in 24 GB; without it, it OOMs).

Training


Base	`urchade/gliner_large-v2.1` — 445.5M params, deberta-v3-large, span mode `markerV0`
Data	200,000 examples / 466,878 entities from `omeryentur/pam-pii-redaction-2m`
Steps	12,500 (1 epoch), batch 16, `max_width` 48, `max_types` 25
LR	encoder 6e-6 / heads 3e-5, cosine, 5% warmup
Precision	bf16 autocast + gradient checkpointing
Hardware	1x RTX 4090, 110.5 min
Loss	154.1 -> 6.8

Trained with a manual training loop, not gliner.training.Trainer: on gliner 0.2.x + transformers 5.x that wrapper silently reports loss=0 and never updates weights. The loop asserts grad_norm > 0 in the first steps and verifies the weight delta at the end.

Limitations

Effective context is 512 tokens, shared between label descriptions and text. With 50 labels only ~310-360 tokens remain for the text.
Training data is ~87% synthetic (LLM-written templates filled by a deterministic generator); expect a drop on real ticket prose.
The negative class is domain-specific: version strings, branch names, Jira keys, ports, CVE ids, PIDs and timestamps are deliberately not redacted. If your domain wants those masked, this is the wrong starting point.
English-vocabulary tokenizer on Turkish-weighted data — Turkish fragments ~2.5x more than English, which costs context and hurts numeric-identifier labels.
Evaluated only on in-distribution held-out data. No out-of-distribution benchmark.

Attribution

Fine-tuned from urchade/gliner_large-v2.1 (Apache-2.0). Training data derives from BTX24/turkish-privacy-pii-ner (CC-BY-4.0), gretelai/gretel-pii-masking-en-v1 (Apache-2.0), mooselab/SDLog (MIT) and ai4privacy/pii-masking-400k (license: other — review its terms before redistributing derivatives).

omeryentur/gliner-pam-pii-large

gliner-pam-pii-large — PAM/Jira PII & secret redaction, 50 labels