Korean enterprise PII detection — fine-tuned EXAONE 4.0 1.2B with a regex layer in front for structured types. Built and used in production by Teeem.ai.kr.
Final score on the 230-prompt eval (hybrid pipeline):
Metric
Value
Precision
0.928
Recall
0.931
F1
0.930
Pass rate
0.800
9 of 12 PII types are at F1 = 1.000 in the hybrid pipeline.
What this is
A two-stage Korean PII detection system designed to be dropped in front of an LLM so you can mask sensitive data before it leaves your perimeter and unmask it on the way back:
user text → [regex layer] → [EXAONE LoRA] → merge → masked text → upstream LLM
↓
mappings
↓
upstream response ← unmask ← [reverse mappings]
The split is deliberate. Structured PII is a regex problem — phone numbers, RRNs, business registration numbers, account numbers, emails, cards. The ML model is reserved for what regex cannot do reliably: Korean person names, free-form addresses, and organization names. This is the same architecture used by AWS Comprehend, GCP DLP, and Microsoft Presidio.
Per-type performance (hybrid, 230-prompt eval)
Type
P
R
F1
Source
ACCOUNT
1.000
1.000
1.000
regex
BRN
1.000
1.000
1.000
regex
EMAIL
1.000
1.000
1.000
regex
HEALTH_INSURANCE
1.000
1.000
1.000
regex
LICENSE
1.000
1.000
1.000
regex
PASSPORT
1.000
1.000
1.000
regex
PHONE
1.000
1.000
1.000
regex
RRN
1.000
1.000
1.000
regex
CARD
0.882
1.000
0.938
regex
NAME
0.899
0.973
0.934
ML
ORGANIZATION
0.885
0.857
0.871
ML
ADDRESS
0.719
0.622
0.667
ML
ADDRESS is the weakest type — it's the only category where the model has to do free-form span identification with no structural anchor. Future iterations should target it with a dedicated address gazetteer or a separate ADDRESS-only adapter.
For the production gateway (with the regex layer wired in front, mask/unmask, session-scoped mappings, optional AES-256-GCM encryption), use the Teeem PII Gateway: packages/pii-gateway/ in the Teeem monorepo. The TypeScript regex implementation is mirrored here in patterns_typescript/.
Self-hosted deployment recipe
The reference deployment runs on AWS ECS with a g4dn.2xlarge GPU host. You can replicate this anywhere with a 16+ GB GPU.
Container layout (two-container task):
exaone-vllm — vLLM 0.6+ serving the model on localhost:8091
gateway-proxy — Node.js process running the regex layer + EXAONE client + mask/unmask pipeline, listening on :8090, forwarding to upstream LLM
Cold-start time: ~3-4 minutes (most of which is downloading the 2.4 GB safetensors). Use a persistent volume / cache directory if you spin the service up and down often.
Base model:LGAI-EXAONE/EXAONE-4.0-1.2BMethod: LoRA (PEFT) — r=32, alpha=64Target modules:q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_projHardware: AWS g6e.xlarge (NVIDIA L40S 48 GB), bf16
Optimizer:adamw_torch_fused, lr 8e-5, batch 4 × grad_accum 2
Steps per iteration: 400
Total iterations: 14
Each iteration: generate fresh augmentation (generate_aug.py) → train on aug + replay buffer → merge LoRA → eval → analyze failures → adjust templates → repeat.
The full training data, replay buffer, scripts, and per-iteration metrics live in the project's S3 bucket — they are not in this HF repo because they contain templated synthetic Korean PII.
Iteration history (highlights)
Iter
F1 (orig 30)
F1 (230)
Notes
baseline (raw EXAONE)
~0.50
—
No fine-tuning, hallucinates types
iter 5
0.84
—
r=16 LoRA, ACCOUNT stuck at 0/3
iter 6
0.86
—
r=32 + MLP targets, ACCOUNT 1/3
iter 7
0.87
—
3/3 on orig 30 — first ACCOUNT win
iter 8
—
0.84
Expanded bank vocab, ACCOUNT 27/37
iter 11
—
0.845
L40S bf16, batch 32 — over-eager EMAIL
iter 12
—
0.69
Disaster: trained from raw HF base, regression on fundamentals
iter 13
0.93
0.85 (raw) / 0.926 (hybrid)
Clean reset; regex layer added
iter 14
0.969
0.930 (hybrid)
ADDRESS-focused refinement; final
The "stuck ACCOUNT" story
For five iterations, ACCOUNT recall sat at 0/3 on the original 30-prompt eval. We thought it was a vocabulary problem, then a regex-vs-NN problem, then a context problem. None of those explained it. The actual cause was LoRA capacity — r=16 with attention-only target modules wasn't enough to learn the digit-pattern → ACCOUNT mapping for novel bank names. Bumping to r=32 and adding the MLP target modules (gate_proj, up_proj, down_proj) unlocked it in one iteration.
The lesson: when a single PII type is stuck while everything else trains fine, don't add more training data — first check whether your adapter has enough capacity to represent the pattern at all.
The "regex breakthrough"
After iter 11, the model was plateauing around F1 ≈ 0.87 on the 230-prompt eval. Each iteration overfit a slightly different bank vocabulary or phone format. We wired in a regex layer purely as a defensive measure — and ACCOUNT recall jumped from 0.703 (26/37) to 1.000 (37/37) in a single rescore, with zero false positives. EMAIL went from 34/42 to 42/42 the same way.
The lesson: this is a hybrid problem, not an ML problem. The structured types didn't need a smarter model; they needed to not be the model's responsibility.
