Views
No views yet
AHVNUMBER (Swiss social security), IBAN, CREDITCARD, PHONE, EMAIL,
FULLNAME, ADDRESS, ZIPCODE, CITY, CANTON, DATEOFBIRTH, DATE,
TIME, AGE, GENDER, ORG, DOCID, PLATE, LANDREGISTRY1from transformers import pipeline
2
3ner = pipeline(
4 "token-classification",
5 model="sandsiv/sandsiv-278-pii-ch",
6 aggregation_strategy="simple",
7)
8ner("Il numero AVS di Anna Bianchi è 756.1234.5678.97.")
9# -> FULLNAME 'Anna Bianchi', AHVNUMBER '756.1234.5678.97'dtype="auto"
from code paths that would cast to fp16 before inference on CPU/MPS.onnx/ folder contains the exported model, verified to match the PyTorch
reference (see evaluation below):| File | Size | Entity-level F1 (val subset) |
|---|---|---|
onnx/model.onnx (fp32) | 1.11 GB | 0.9991 (identical to PyTorch) |
onnx/model-int8.onnx (dynamic int8) | 338 MB | 0.9978 (−0.0013 vs PyTorch) |
onnxruntime + tokenizers (no PyTorch/transformers).
Session inputs: input_ids, attention_mask. Outputs: per-token logits
(39 BIO labels — see config.json id2label).1import numpy as np
2import onnxruntime as ort
3from tokenizers import Tokenizer
4
5tok = Tokenizer.from_file("tokenizer.json") # from onnx/ or the repo root
6tok.enable_truncation(128)
7tok.enable_padding()
8
9sess = ort.InferenceSession("onnx/model-int8.onnx")
10enc = tok.encode_batch(["Il numero AVS di Anna Bianchi è 756.1234.5678.97."],
11 is_pretokenized=False)
12logits = sess.run(["logits"], {
13 "input_ids": np.array([e.ids for e in enc], dtype=np.int64),
14 "attention_mask": np.array([e.attention_mask for e in enc], dtype=np.int64),
15})[0]ch-pii-synth-0.3.0 + targeted supplement),
no real personal data was used. 29'000 examples, balanced across the 4
languages (7'250 each), 232+ templates, split 90/5/5 stratified by language.| Metric | Value |
|---|---|
| Test set entity-level F1 (seqeval) | 0.9988 |
| — per language: de / en / fr / it | 1.0000 / 0.9988 / 0.9988 / 0.9975 |
| Hand-written out-of-template realistic set (26 sentences) | F1 0.964 (P 0.947, R 0.982) |
S1234567A) may be
split or partially mislabeled as PLATE.aggregation_strategy="simple";
merging adjacent same-label spans fixes this.AGE is the weakest entity type (test F1 ≈ 0.95).