Views
No views yet
redacted_text and entities. Optimized to run locally with strong accuracy and strict schema adherence.google/gemma-3-270m1{
2 "redacted_text": "Text with in-place tokens",
3 "entities": [
4 {"value": "<original>", "replacement_token": "[TOKEN]", "reason": "<why>"}
5 ]
6}[PERSON] [EMAIL] [PHONE] [ADDRESS] [SSN] [ID] [UUID] [CARD_LAST4:####] [IBAN_LAST4:####] [GENDER] [AGE] [RACE] [MARITAL_STATUS](value, replacement_token) pairs (reason/order ignored). Score: 0.73 +/- 0.07.1SYSTEM_PROMPT = """
2You are a problem solving model working on task_description XML block:
3<task_description>
4Produce a redacted version of texts, removing sensitive personal data while preserving operational signals. The model must return a single json blob with:
5
6* **redacted_text** is the input with minimal, in-place replacements of redacted entities.
7* **entities** as an array of objects with exactly three fields {value: original_value, replacement_token: replacement, reason: reasoning}.
8
9## What to redact (→ replacement token)
10
11* **PERSON** — customer/patient/person names (first/last/full; identifying initials) → `[PERSON]`
12* **EMAIL** — any email, including obfuscated `name(at)domain(dot)com` → `[EMAIL]`
13* **PHONE** — any international/national format (separators/emoji bullets allowed) → `[PHONE]`
14* **ADDRESS** — street + number; full postal lines; apartment/unit numbers → `[ADDRESS]`
15* **SSN** — US Social Security numbers → `[SSN]`
16* **ID** — national IDs (PESEL, NIN, Aadhaar, DNI, etc.) when personal → `[ID]`
17* **UUID** — person-scoped system identifiers (e.g., MRN/NHS/patient IDs/customer UUIDs) → `[UUID]`
18* **CREDIT_CARD** — 13–19 digits (spaces/hyphens allowed) → `[CARD_LAST4:####]` (keep last-4 only)
19* **IBAN** — IBAN/bank account numbers → `[IBAN_LAST4:####]` (keep last-4 only)
20* **GENDER** — self-identification (male/female/non-binary/etc.) → `[GENDER]`
21* **AGE** — stated ages (“I’m 29”, “age: 47”, “29 y/o”) → `[AGE_YEARS:##]`
22* **RACE** — race/ethnicity self-identification → `[RACE]`
23* **MARITAL_STATUS** — married/single/divorced/widowed/partnered → `[MARITAL_STATUS]`
24
25
26## Keep (do not redact)
27
28* Card **last-4** when only last-4 is present (e.g., “ending 9021”, “•••• 9021”).
29* Operational IDs: order/ticket/invoice numbers, shipment tracking, device serials, case IDs.
30* Non-personal org info: company names, product names, team names.
31* Cities/countries alone (redact full street+number, not plain city/country mentions).
32
33## Output schema (exactly these fields)
34* **redacted_text** The original text with all the sensitive information replaced with redacted tokens
35* **entities** Array with all the replaced elements, each element represented by following fields
36 * **replacement_token**: one of `[PERSON] | [EMAIL] | [PHONE] | [ADDRESS] | [SSN] | [ID] | [UUID] | [CREDIT_CARD] | [IBAN] | [GENDER] | [AGE] | [RACE] | [MARITAL_STATUS]`
37 * **value**: original text that was redacted
38 * **reason**: brief string explaining the rule/rationale
39
40for example
41{
42 "redacted_text": "Hi, I'm [PERSON] and my email is [EMAIL].",
43 "entities": [
44 { "type": "PERSON", "value": "John Smith", "reason": "person name"},
45 { "type": "EMAIL", "value": "john.smith@example.com", "reason": "email"},
46 ]
47}
48</task_description>
49You will be given a single task with context in the context XML block and the task in the question XML block
50Solve the task in question block based on the context in context block.
51Generate only the answer, do not generate anything else
52"""
53
54PROMPT_TEMPLATE = """
55
56Now for the real task, solve the task in question block based on the context in context block.
57Generate only the solution, do not generate anything else
58<context>
59{context}
60</context>
61<question>Redact provided text according to the task description and return redacted elements.</question>
62"""
63
64from openai import OpenAI
65
66PORT = "PORT GOES HERE" # 8000 for vllm, 11434 for ollama
67MODEL_NAME = "NAME USED FOR SETTING UP THE CLIENT"
68TEXT_TO_REDACT = "NI number AB123456C confirmed."
69
70client = OpenAI(base_url=f"http://127.0.0.1:{PORT}/v1", api_key="EMPTY")
71chat_response = client.chat.completions.create(
72 model=MODEL_NAME,
73 messages=[
74 {"role": "system", "content": SYSTEM_PROMPT},
75 {"role": "user", "content": PROMPT_TEMPLATE.format(context=TEXT_TO_REDACT)},
76 ],
77 temperature=0,
78)