A LoRA adapter for INSAIT-Institute/BgGPT-Gemma-3-4B-IT, fine-tuned to detect and mask personal data (PII) in free-form Bulgarian text and return the result as structured JSON.

It is the AI component of a larger hybrid anonymizer that pairs this model with a deterministic regex + checksum layer (for ЕГН, IBAN, phone, email). 👉 Full project & code: https://github.com/akaraangov/bg-gdpr-anonymizer

What it does

Given Bulgarian text, the model returns JSON like:

json

{
  "masked_text": "Казвам се [ИМЕ], живея в [ГРАД] на [АДРЕС].",
  "extracted_entities": {"ИМЕ": ["..."], "ГРАД": ["..."], "АДРЕС": ["..."]}
}

Tags: ИМЕ, ГРАД, АДРЕС, ЕГН, IBAN, ТЕЛЕФОН, ИМЕЙЛ, РЕГ_НОМЕР.

In the hybrid pipeline the model focuses on names, cities and addresses, while IDs with a checksum (ЕГН, IBAN) and strict-format data (phone, email) are handled by the deterministic regex layer.

Usage

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE = "INSAIT-Institute/BgGPT-Gemma-3-4B-IT"      # or unsloth/gemma-3-4b-it
ADAPTER = "akaraangov/bggpt-gdpr-lora-v2"

tok = AutoTokenizer.from_pretrained(ADAPTER)
base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER).eval()

sys = ("Ти си асистент за анонимизация на лични данни (PII) в български текст "
       "съгласно GDPR. Връщаш само валиден JSON с полета \"masked_text\" и "
       "\"extracted_entities\".")
text = "Анонимизирай следния български текст по GDPR стандарти: 'Иван Петров от Пловдив се обади.'"
msgs = [{"role": "user", "content": sys + "\n\n" + text}]

inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                                 return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Training

Method: QLoRA (4-bit NF4), LoRA on the attention & MLP projections.
Data: ~5,400 synthetic Bulgarian examples in ShareGPT format, generated algorithmically. PII appears with and without trigger words, in Bulgarian and international phone formats, across multi-person documents, plus negative samples with non-PII numbers — so the model learns structure, not cue words.
All identifiers in the training data are algorithmically valid but entirely fictional; no real personal data was used.

Limitations

Trained on synthetic data — a strong demonstrator, not a certified compliance tool. Always review the output.
For production use, pair it with the regex layer from the project; do not rely on the model alone for ЕГН/IBAN/phone/email.

License

Governed by the Gemma Terms of Use (inherited from the base model). Project code is MIT-licensed.