Views
No views yet
openai/privacy-filter
for fine-grained PII extraction across 54 categories in 16 languages.openai/privacy-filter — 1.4B-parameter MoE (50M active per token), BIOES token-classification headpii-masking-200k, pii-masking-400k, and open-pii-masking-500k-ai4privacy, language-balancedopf train (OpenAI's official fine-tuning CLI) — full fine-tune, AdamW, balanced language sampling, 5 epochs, bf16O + 54 × B/I/E/S)Family at a glance. Same architecture, three runtimes:
- PyTorch (this repo) — CPU + CUDA, anywhere transformers runs.
- MLX BF16 —
OpenMed/privacy-filter-multilingual-mlx— Apple Silicon, full precision.- MLX 8-bit —
OpenMed/privacy-filter-multilingual-mlx-8bit— Apple Silicon, smaller + faster.
extract_pii() / deidentify() with built-in BIOES Viterbi
decoding, span refinement, and a Faker-backed obfuscation engine. Same call
on every host — Apple Silicon picks up MLX automatically; everywhere else uses
this PyTorch checkpoint.pip install -U "openmed[hf]"1from openmed import extract_pii, deidentify
2
3text = (
4 "Patient Sarah Johnson (DOB 03/15/1985), MRN 4872910, "
5 "phone 415-555-0123, email sarah.johnson@example.com."
6)
7
8# Extract grouped entity spans
9result = extract_pii(text, model_name="OpenMed/privacy-filter-multilingual")
10for ent in result.entities:
11 print(f"{ent.label:30s} {ent.text!r} conf={ent.confidence:.2f}")
12
13# De-identify with any of the supported methods
14masked = deidentify(text, method="mask", model_name="OpenMed/privacy-filter-multilingual")
15removed = deidentify(text, method="remove", model_name="OpenMed/privacy-filter-multilingual")
16hashed = deidentify(text, method="hash", model_name="OpenMed/privacy-filter-multilingual")
17
18# Faker-backed locale-aware obfuscation, deterministic with consistent=True+seed
19fake = deidentify(
20 text,
21 method="replace",
22 model_name="OpenMed/privacy-filter-multilingual",
23 consistent=True,
24 seed=42,
25)
26print(fake.deidentified_text)OpenMed/privacy-filter-multilingual-mlx* model names also work in the same
extract_pii() / deidentify() calls — on a non-Apple-Silicon host they
automatically fall back to this PyTorch checkpoint with a one-time warning.
So you can ship MLX names in code and still run on Linux/Windows.trust_remote_code=True for you, runs the model's
own BIOES Viterbi decoder, and skips OpenMed's regex smart-merging (the model
already produces clean spans).| Category | Typical examples |
|---|---|
| Identity | FIRSTNAME, MIDDLENAME, LASTNAME, PREFIX, AGE, GENDER, SEX, EYECOLOR, HEIGHT, USERNAME, OCCUPATION, JOBTITLE, JOBDEPARTMENT, ORGANIZATION, USERAGENT |
| Contact | EMAIL, PHONE, URL |
| Address | STREET, BUILDINGNUMBER, SECONDARYADDRESS, CITY, COUNTY, STATE, ZIPCODE, GPSCOORDINATES, ORDINALDIRECTION |
| Dates & time | DATE, DATEOFBIRTH, TIME |
| Government IDs | SSN |
| Financial | ACCOUNTNAME, BANKACCOUNT, IBAN, BIC, CREDITCARD, CREDITCARDISSUER, CVV, PIN, MASKEDNUMBER, AMOUNT, CURRENCY, CURRENCYCODE, CURRENCYNAME, CURRENCYSYMBOL |
| Crypto | BITCOINADDRESS, ETHEREUMADDRESS, LITECOINADDRESS |
| Vehicle | VIN, VRM |
| Digital | IPADDRESS, MACADDRESS, IMEI |
| Auth | PASSWORD |
O plus B-, I-, E-, S- for each of the 54 categories
(4 × 54 + 1 = 217). The id2label mapping is shipped with the model.opf's default "copy-from-matching-base" head init.
Of the 217 new BIOES classes, the few with exact base-vocabulary matches
(O, B/I/E/S-account_name, etc.) were copied directly; the rest were copied
from semantically-adjacent coarse rows and fine-tuned end-to-end.opf training/eval CLI). Everything in
this repo is a fine-tune on top of that release.pii-masking-200k,
pii-masking-400k,
open-pii-masking-500k-ai4privacy.transformers /
huggingface_hub ecosystem this model ships through.1@misc{openmed_privacy_filter_multilingual_2026,
2 author = {OpenMed},
3 title = {{OpenMed/privacy-filter-multilingual}: multilingual fine-grained PII extraction across 16 languages and 54 categories},
4 year = {2026},
5 publisher = {Hugging Face},
6 howpublished = {\url{https://huggingface.co/OpenMed/privacy-filter-multilingual}}
7}
8
9@misc{openmed_2026,
10 author = {OpenMed},
11 title = {{OpenMed}: open models and resources for healthcare NLP},
12 year = {2026},
13 publisher = {Hugging Face},
14 howpublished = {\url{https://huggingface.co/OpenMed}}
15}
16
17@misc{openai_privacy_filter_2025,
18 author = {OpenAI},
19 title = {{openai/privacy-filter}},
20 year = {2025},
21 publisher = {Hugging Face},
22 howpublished = {\url{https://huggingface.co/openai/privacy-filter}}
23}
24
25@misc{ai4privacy_pii_masking,
26 author = {AI4Privacy},
27 title = {{AI4Privacy PII Masking Datasets}},
28 publisher = {Hugging Face},
29 howpublished = {\url{https://huggingface.co/ai4privacy}}
30}