MoAI-Privacy-Filter is a Korean and English privacy-related entity detection model built by full fine-tuning openai/privacy-filter. It recognizes 29 entity types and emits 117 BIOES token classes. The training data emphasizes financial services and customer-service/VOC text while also covering identity, security, and infrastructure scenarios.
The model detects entity spans but does not decide how they should be masked or retained. Applications can apply their own handling policy to each predicted label. This distinction is especially important for PORT and ORGANIZATION, which are non-PII disambiguation labels included in the output taxonomy.
On held-out validation, strict micro F1 is 0.9824 for ko and 0.9708 for en. On an independently generated Golden Set, strict micro F1 is 0.9732 for ko and 0.9650 for en.
Approximately 1.4B-parameter MoE, 8 layers, hidden size 640, 128 local experts, top-4 expert routing
Task
Token classification with BIOES span boundaries
Languages
Korean and English
Primary domains
Financial services, customer service/VOC, identity, security, and infrastructure
Entity labels
29
Output classes
117 - O plus four BIOES classes for each entity label
Training sequence limit
1024 tokens before special tokens
Artifact format
BF16 safetensors with 8 attention sinks tensors retained in FP32
Tested software
Transformers 5.13.1 and PyTorch 2.13.0
License
Apache 2.0
The model version and dataset version use independent version numbers. This model is v3 and was trained on dataset v1.
2. Label Taxonomy
Label
Definition
PERSON
Full personal name as one span.
RRN
Korean resident registration number.
FRN
Korean foreign resident registration number.
SSN
Social-security-number family inherited from ai4privacy SOCIALNUM; not limited to the US 9-digit form.
GENERIC_ID
Identity-card or tax identifier that cannot be assigned to a more specific country-level label.
CARD_NUMBER
Credit or debit card PAN.
ACCOUNT_NUMBER
Bank account number.
SECRET
Password, API key, access token, or similar authentication secret.
USER_ID
Online account or member identifier.
EMAIL
Email address, including intentionally obfuscated forms represented in the training data.
PHONE
Mobile or landline telephone number, including intentionally verbalized forms represented in the training data.
PASSPORT
Passport number.
DRIVER_LICENSE
Driver's license number.
ADDRESS
City, street, and building components represented as one address span.
ZIPCODE
Postal code kept separate from ADDRESS.
DATE
Date or time. Timezone-only strings are not included in this label.
CARD_EXPIRY
Payment-card expiration date.
CVC
Card verification code.
IPIN
Korean I-PIN identifier.
TRANSACTION_APPROVAL_ID
Payment authorization or transaction approval identifier.
BUSINESS_ID
Business registration number or merchant identifier.
VIRTUAL_CARD_NUMBER
Alternate or virtual card number.
CI
Korean identity-linkage information value.
IPADDRESS
IPv4 network address.
MACADDRESS
48-bit MAC address.
IMEI
Mobile-equipment identifier.
PORT
Network service port from 0 to 65535; a non-PII disambiguation label.
ORGANIZATION
Company, bank, hospital, or other organization name; a non-PII disambiguation label.
URL
Full web URL, including path, query, and fragment when present.
Each entity label has B-, I-, E-, and S- boundary classes. Together with O, the model therefore has 4 x 29 + 1 = 117 output classes. O means that the model predicts no taxonomy entity at that token; it does not guarantee that the surrounding text is non-sensitive.
Apply constrained BIOES Viterbi decoding to logits[0], then map the decoded token spans to the original text with offsets. Character-span records can then be represented in the following form.
Character offsets use Python's half-open interval [start, end). A downstream application could render those spans as follows, but this replacement behavior is not part of the model.
고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다.
3.1. Decoding
The reported metrics use constrained Viterbi decoding over the BIOES transition grammar, followed by whitespace boundary refinement. Independent per-token argmax can emit invalid BIOES sequences and is not the reported decoding path.
The bundled viterbi_calibration.json contains six transition biases. The default operating point sets all biases to zero, so BIOES transition constraints remain active without an additional precision-recall adjustment. Convert logits to FP32 before decoding.
Use a decoder that implements this BIOES constraint contract. The upstream openai/privacy-filter project provides the reference implementation and decoding behavior on which this model is based.
For batches, use right padding and pass only input_ids and attention_mask to the model. Keep offset_mapping outside the model for character-span reconstruction. Inputs longer than 1024 tokens were not represented in the training regime and should be chunked with enough overlap for the target use case.
The Golden Set was not used for training, checkpoint selection, or calibration. The training and validation data contain all 29 labels in both languages. English represents 25.74% of train and 26.03% of validation.
The dataset combines relabeled rows from ai4privacy/pii-masking-openpii-1.5m with Korean perturbation, English replay, and statically authored synthesis rows. Synthesis covers positive, confusion, hard-negative, weak-cue, long-context, and multi-label scenarios. The dataset is designed as synthetic training data and contains no operational customer records.
4.2. Procedure
Training loss, learning rate, and gradient norm by training step
Item
Value
Method
Full fine-tuning of all parameters, including experts and router
Classification head
33 base classes expanded to 117 classes through taxonomy-aware initialization
Loss
Token-level cross-entropy
Sequence length
Maximum 1024 tokens before special tokens
Batch
Per-device 16, effective 16 on one GPU
Learning rate
1e-4 with linear decay
Warmup
543 steps, approximately 3% of 18,080 total steps
Optimizer
Fused AdamW, weight decay 0.0, maximum gradient norm 1.0
Epochs
5, with the best checkpoint selected by validation strict micro F1
Precision
FP32 master parameters with BF16 autocast; MoE router and expert compute protected in FP32
Seed
42 for training and data sampling
Hardware
1 NVIDIA H100 80 GB GPU
The selected FP32 checkpoint was exported as a deployment artifact in BF16. Artifact validation found 132 BF16 tensors and 8 FP32 attention sinks tensors.
4.3. Included Files
File
Purpose
model.safetensors
BF16 deployment weights with FP32 attention sinks.
config.json
Architecture and 117-class label mapping.
tokenizer.json and tokenizer_config.json
Tokenizer files associated with the base model.
label-taxonomy.yaml
Label definitions and source-to-target mapping used for training.
viterbi_calibration.json
Constrained-decoding operating-point biases.
5. Evaluation
5.1. Setup
The validation split was used for checkpoint selection and experiment comparison. Final generalization was measured on a separately generated Golden Set containing weak-context entities, surface-similar decoys, label-confusion pairs, and boundary variants.
The headline metrics are language-slice strict micro Precision, Recall, and F1. A predicted entity is correct only when both its label and complete span boundary match the reference. Golden evaluation uses constrained Viterbi decoding and whitespace boundary refinement.
5.2. Results
Validation precision, recall, micro F1, and macro F1 by training step
Language
Validation P
Validation R
Validation F1
Golden P
Golden R
Golden F1
F1 Difference
ko
0.9829
0.9819
0.9824
0.9735
0.9729
0.9732
-0.92%p
en
0.9708
0.9708
0.9708
0.9652
0.9649
0.9650
-0.58%p
F1 Difference is Golden F1 minus Validation F1. The smaller Golden scores indicate a limited generalization decrease of 0.92 percentage points for Korean and 0.58 percentage points for English.
5.3. Error Characteristics
The aggregate results do not mean that every label performs equally. Error analysis of the English Golden slice shows the most visible weaknesses in ACCOUNT_NUMBER, ZIPCODE, PORT, and ORGANIZATION. Frequent confusion directions include PORT versus ZIPCODE, ACCOUNT_NUMBER versus BUSINESS_ID or IPIN, and ORGANIZATION versus PERSON.
These patterns are consistent with labels that share numeric shapes or require contextual role information. Downstream systems should evaluate per-label behavior on their own traffic, especially when label identity changes the handling action.
6. Intended Use
Suitable uses include:
Detecting privacy-related entities before Korean or English text is sent to an LLM or another downstream service.
Supporting offline privacy review of customer-service text, documents, email, and logs.
Producing typed entity spans for an application-specific masking, routing, retention, or review policy.
The model is not a complete anonymization system, a legal-compliance guarantee, or a substitute for domain-specific review. It should not be used as the sole control for high-impact decisions. Applications remain responsible for deciding whether each detected label is masked, transformed, retained, or escalated.
7. Limitations
Synthetic evaluation - Training, validation, and Golden data are synthetic. Performance on real customer text, OCR noise, slang, novel obfuscation, and unseen document structures has not been established.
Label-specific variation - High aggregate F1 can hide weaker labels and confusion pairs. ACCOUNT_NUMBER, ZIPCODE, PORT, and ORGANIZATION require particular attention based on the current Golden analysis.
Non-PII labels - PORT and ORGANIZATION are deliberately predicted even though they are not PII. Consumers must not assume that every non-O label requires the same action.
Context and boundary sensitivity - Weak contextual evidence, shared numeric formats, and long entity boundaries can produce missed entities, boundary errors, or label swaps.
Long inputs - The base architecture supports a larger context, but training examples were limited to 1024 tokens and the observed dataset maximum was 801 tokens. Longer inputs require separate validation and should normally be chunked.
Registry-backed identifiers - Synthetic account, telephone, passport, user, and social-security values cannot be exhaustively checked against private issuance registries. Any coincidental match with a real value is unintended.
Language and domain scope - Evaluation covers Korean and English with emphasis on financial, customer-service/VOC, identity, security, and infrastructure contexts. Other languages and domains are unsupported.
8. License, Attribution, and Citation
The model is released under the Apache 2.0 license. Its training dataset is released under CC BY 4.0 and is derived from ai4privacy/pii-masking-openpii-1.5m; follow the dataset card for its attribution requirements.
bibtex
1@misc{bccard2026moaiprivacyfilterv3,
2 title = {MoAI-Privacy-Filter v3: Korean and English Privacy-Related Entity Detection},
3 author = {BC Card},
4 year = {2026},
5 howpublished = {https://huggingface.co/BCCard/MoAI-Privacy-Filter},
6 note = {Full fine-tune of openai/privacy-filter on BCCard/privacy-filter-openpii-masking v1}
7}
This model is provided as is, without warranties of accuracy, completeness, non-infringement, or fitness for a particular purpose. Users are responsible for testing the model in their own environment and ensuring that its use complies with applicable laws, regulations, contractual obligations, and organizational policies.
